tittat
Active Member
- Messages
- 2,478
- Reaction score
- 1
- Points
- 38
Googlebot received an update that permits it to complete certain forms, and learn more about the site hosting them. Websites place content behind forms for the purpose of collecting information from a visitor requesting access to it.
here is link to the googles official blog...
http://googlewebmastercentral.blogspot.com/2008/04/crawling-through-html-forms.html
Only a small number of particularly useful sites receive this treatment, and our crawl agent, the ever-friendly Googlebot, always adheres to robots.txt, nofollow, and noindex directives. That means that if a search form is forbidden in robots.txt, we won't crawl any of the URLs that a form would generate. Similarly, we only retrieve GET forms and avoid forms that require any kind of user information.
here is link to the googles official blog...
http://googlewebmastercentral.blogspot.com/2008/04/crawling-through-html-forms.html