X-Robots noindex specific page in .htaccess

Question

Can I 'noindex, follow' a specific page using x robots in .htaccess?

I've found some instructions for noindexing types of files, but I can't find instruction to noindex a single page, and what I have tried so far hasn't worked.

This is the page I'm looking to noindex:

http://www.examplesite.com.au/index.php?route=news/headlines

This is what I have tried so far:

<FilesMatch "/index.php?route=news/headlines$">
 Header set X-Robots-Tag "noindex, follow"
</FilesMatch>

Thanks for your time.

Toadflax

FilesMatch argument is a regular expression, so you probably must "escape" some special characters like . and ?. I can't test it right now, but I would like to test "/index\.php\?route=news/headlines$". – Wiburg

Handling this case would be much easier in PHP than in .htaccess; can't you just call header('X-Robots-Tag: noindex, follow'), probably in your headlines action in your news controller (if this is indeed a MVC architecture)? – Ordinal

Answer 1

It seems to be impossible to match the request parameters from within a .htaccess file. Here is a list of what you can match against: http://httpd.apache.org/docs/2.2/sections.html

It will be much easier to do it in your script. If you are running on PHP try:

header('X-Robots-Tag: noindex, follow');

You can easily build conditions on $_GET, REQUEST_URI and so on.

Nessie

Answer 2

RewriteEngine on
RewriteBase /

#set env variable if url matches
RewriteCond %{QUERY_STRING} ^route=news/headlines$
RewriteRule ^index\.php$ - [env=NOINDEXFOLLOW:true]

#only sent header if env variable set
Header set X-Robots-Tag "noindex, follow" env=NOINDEXFOLLOW

FilesMatch works on (local) files, not urls. So it would try to match only the /index.php part of the url. <location> would be more appropriate, but as far as I can read from the documentation, querystrings are not allowed here. So I ended up with the above solution (I really liked this challenge). Although php would be the more obvious place to put this, but that is up to you.

The solution requires mod_rewrite, and mod_headers of course.

Mcclendon

Answer 3

Note that you'll need the mod_headers module enabled to set the headers.

Though like others have said, it seems better to use the php tag. Does that not work?

Dictator

Answer 4

According to Google the syntax would be a little different:

<Files ~ "\.pdf$">
  Header set X-Robots-Tag "noindex, nofollow"
</Files>

https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag

Jenny