Form Crawling Program by google..

tittat

Active Member
Messages
2,478
Reaction score
1
Points
38
Googlebot received an update that permits it to complete certain forms, and learn more about the site hosting them. Websites place content behind forms for the purpose of collecting information from a visitor requesting access to it.


Only a small number of particularly useful sites receive this treatment, and our crawl agent, the ever-friendly Googlebot, always adheres to robots.txt, nofollow, and noindex directives. That means that if a search form is forbidden in robots.txt, we won't crawl any of the URLs that a form would generate. Similarly, we only retrieve GET forms and avoid forms that require any kind of user information.




here is link to the googles official blog...
http://googlewebmastercentral.blogspot.com/2008/04/crawling-through-html-forms.html
 
Top