Google bot. Anyone faced problems?

[jensen]

Just got an email from a friend warning me about the dangers of Google bot. It seemed that a website got deleted because a google bot visited the site and wipoed out the site. It was because of some script that the bot craled.

Is this a major danger?

 

[bigguy]

No, I really can`t see this happening at all.

 

[Bryon]

If you have a script on yoru server that is in your web root (public_html) and can cause harm to your site if it's ran, then it's pretty much your fault. You should always keep things like that below your web root directory.

You could do what is described here though to help if you come across problems:
http://google.com/webmasters/bot.html

Overall though, like Geoff said, there isn't much to worry about unless you have links to "hidden files" floating around.

 

[Nathan]

MSNBot - 48
Googlebot - 19
Inktomi Slurp - 5
Unknown robot (identified by 'crawl') - 2


Information provided by AwStats on http://dewayne-smith.com

The latest all bots crawled my site was yesterday.

 

[Spartan Erik]

lol good grief, if you're that paranoid of a google bot (which you shouldnt be since google bots only want to see what pages you've got), you can set up a robots.txt file to block them..

ive set it to block them all.. if you need a simple tut PM me

 

[echo_unlimited]

Why did you block them?

 

[Bryon]

You could easily block all bots from crawling you pages by placing this at the top of everyone. (Or by using a robots.txt file.. )

<meta name="robots" content="noindex,nofollow">

Why though? :-\

 

[Conor]

If you want traffic for your site don't worry about spiders.

 

[jensen]

Thanks for the assurance guys. I surely want the bots to crawl my site so that it would be found easily via google. Hmm.. yeah why won't you want the bots on your site? (since deletion is not a problem)