How can I stop bots from following certain links on my site

[csc2ya]

I'm trying to figure out what to put in the robots.txt file to stop bots from following certain links on my site. I sometimes find certain quotes voted up or down, and on checking the votes table, I usually find search engine bots have followed the vote links.

I know it's a nofollow directive, but i'm unsure exactly what to put in the file to stop all bots from following the voting links.

Examples of the vote links used are as follows:

Up: http://csc2ya.co.cc/?do=rate&q=1000&r=good
Down: http://csc2ya.co.cc/?do=rate&q=1000&r=bad

(Vote links are valid apart from the quote numbers)

Thanks in advance.

 

[ah-blabla]

There's a good guide here. However not all bots will keep to what you ask, so be warned that it might not end everything. And I'm not sure about wildcards in the path you specify though.

 

[descalzo]

I'm trying to figure out what to put in the robots.txt file to stop bots from following certain links on my site.

From my understanding, there is no way to do it in a robots.txt file for the kind of link you are talking about.

Note also that globbing and regular expression are not supported in either the User-agent or Disallow lines.
The '*' in the User-agent field is a special value meaning "any robot".
Specifically, you cannot have lines like "User-agent: *bot*", "Disallow: /tmp/*" or "Disallow: *.gif".

As noted above, bots do not have to honor your robots.txt file anyway.

That is why using a form with POST is a 'better' method than links for actions where you do not want 'automatic' visits.

 

[ah-blabla]

From my understanding, there is no way to do it in a robots.txt file for the kind of link you are talking about.

If you list every single link you don't want visited it might work. However, from what I have heard, google at least, still follow the link, but don't index it.

 

[csc2ya]

Thanks for the responses. I found I already had a robots.txt file that I forgot about so i've changed it to stop all robots following any links.

As you say, it may not work all the time, but even minimizing the effect they're currently having is better than nothing.

Unfortunately, I can't change the links to forms with posts, as the site runs on a pre-built script I got from another site, and I think changing them would just cause more problems in the future.