How do I modify robots.txt in Plone?

Question

I've got a Plone site that I administer and I'd like to add some pages to the Disallow of a robots.txt.

It appears that Plone automatically generates a robots.txt file. I can't find any way to modify that. I've also tried adding a 'robots.txt' file to the root of the app, but it says that "robots.txt is reserved"

Does anyone know how to modify this with Plone (v3)?

Millsap

Answer 1

1. Create a robots.txt on your desktop
2. Go in the xmi of the plone site -> add a new file -> call it robot.txt
3. Choose it from the desktop

source: http://plone.org/support/forums/general#nabble-td329779

Thermoscope

Yeah, I tried that, as noted above, but it comes back with an error of "robots.txt is reserved". The post mentioned above is from 2007, which is perhaps for Plone 2.x. I believe Plone 3.x added "robots.txt" functionality See the "3.0-alpha2" release notes plone.org/products/plone/releases/3.0 "Added robots.txt file. This should remove some unwanted not-found-errors and give fresh Plone-admins something to customise for robots if they want it." – Millsap

Nevermind. The steps listed didn't work for me, but the link provided did indeed contain the answer: "in ZMI, add a DTMLDocument object, call it robots.txt and add your statements. " – Millsap

This is long out of date. Use newbery's suggestions. – Glyptodont

Answer 2

As has been noted already, Plone 3.x already includes a robots.txt file so the preferred solution now is probably NOT to create a new robots.txt in the ZMI but rather to customize the included one.

Using the ZMI, go to /portal_skins/plone_templates/ and find the robots.txt file. Customize as you would any other skin element.

Or using the filesystem, just add your robots.txt file to a custom skin product.

Perrotta

Answer 3

In Plone 5 it was turned into a BrowserView, so now you can edit it in the Site Settings at /@@site-controlpanel.

Sou