Help with robots.txt!

[divebuddha]

hi there everyone

after a few months of having the web site up and running i wanted to ask a few questions about

about whether the use of robots.txt within my web page enviroment will help with the broswers like google

to index my pages easier or will it be more hassle than its worth allowing all the user agents to visit my

pages?

ive tried to get all the meta txts in the corect places but again if anyone want to take a look and tell me

if im doing anything wrong and how to put it write i will only be too glad to take that info onboard?

also do you allow all bots

or do you only allow some?

and do yu dissallow the bots to visit certain parts of your site?

any help would be apreciated?


http://www.divebuddha.x10hosting.com

 

[Zubair]

Signup for Google webmaster tools. http://www.google.com/webmasters/

It will help you alot

 

[divebuddha]

Signup for Google webmaster tools. http://www.google.com/webmasters/

It will help you alot

i signed up the other day but couldnt make heads or tails of it lol!:dunno:

think i may go and have another look as its been mentioned!

 

[Zubair]

Add your site to Google web master tools. Then confirm your domain by uploading file given by google.

Then go in crawler access option and generate your robot.txt there

 

[divebuddha]

Add your site to Google web master tools. Then confirm your domain by uploading file given by google.

Then go in crawler access option and generate your robot.txt there

yes went through 6that process last night and was very supprised how easy it was to get the info i needed ive just got to create the robot.txt file now then i will see what it brings

the site has been indexed as its a strange name i.e dive buddha