Asterisk in robots.txt

Question

Wondering if following will work for google in robots.txt

Disallow: /*.action

I need to exclude all urls ending with .action.

Is this correct?

Answer 1

I don't think it will work, you would need to move all .action files to a location which you then disallow

Answer 2

The robots.txt specification provides no way to include wildcards, only the beginning of URIs.

Google implement non-standard extensions, described in their documentation (look in the Manually create a robots.txt file section under "To block files of a specific file type").

Answer 3

To block files of a specific file type (for example, .gif), use the following:

User-agent: Googlebot
Disallow: /*.gif$

So, you are close. Use Disallow: /*.action$ with a trailing "$"

Of course, that's merely what Google suggests: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

All bots are different.