Google sitemap files for Rails projects

Question

Is there an easy way to create a sitemaps file for Rails projects? Especially for dynamic sites (such as Stack Overflow for example) there should be a way to dynamically create a sitemaps file. What is the way to go in Ruby and/or Rails?

What would you suggest? Is there any good gem out there?

Matos

Thanks for your question, I got curious about sitemaps and discovered a lot of (other) things to improve my site with respect to Google – Batangas

Answer 1

Add this route towards the bottom of your config/routes.rb file (more specific routes should be listed above it):

map.sitemap '/sitemap.xml', :controller => 'sitemap'

Create the SitemapController (app/controllers/sitemap_controller):

class SitemapController < ApplicationController
  layout nil

  def index
    headers['Content-Type'] = 'application/xml'
    last_post = Post.last
    if stale?(:etag => last_post, :last_modified => last_post.updated_at.utc)
      respond_to do |format|
        format.xml { @posts = Post.sitemap } # sitemap is a named scope
      end
    end
  end
end

—As you can see, this is for a blog, so is using a Post model. This is the HAML view template (app/views/sitemap/index.xml.haml):

- base_url = "http://#{request.host_with_port}"
!!! XML
%urlset{:xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9"}
  - for post in @posts
    %url
      %loc #{base_url}#{post.permalink}
      %lastmod=post.last_modified
      %changefreq monthly
      %priority 0.5

That's it! You can test it by bringing up http://localhost:3000/sitemap.xml (if using Mongrel) in a browser, or perhaps by using cURL.

Note that the controller uses the stale? method to issue a HTTP 304 Not Modified response if there are no new posts sinces the sitemap was last requested.

Elroyels

can someone add clarification on the sitemap named scope, and how setting Last_modified to the time when Post.last was modified helps with checking for staleness? – Bethsaida

@pef, why not? I am doing it know for Rails 3.2, it seems okay. – Subak

Updated John Topley's answer for Rails 4. See new answer below. – Galbraith

Answer 2

Now for rails3, it is better off using full-featured sitemap_generator gem.

Mackler

unfortunately the link you posted does not work. did you mean github.com/adamsalter/sitemap_generator ? – Matos

This is good if your sitemap gets too big. It will split and compress for you according to sitemap limitations. It is also super fast. When I switched from xml builder, I cut the time from 15 minutes to 1.5 minutes. – Cypress

I've used this for my project (jobsite) and this works very well. Thanks! – Riba

Answer 3

I love John Topley's answer because it is so simple and elegant, without the need for a gem. But it's a bit dated, so I've updated his answer for Rails 4 and Google Webmaster Tools' latest sitemap guidelines.

config/routes.rb:

get 'sitemap.xml', :to => 'sitemap#index', :defaults => { :format => 'xml' }

app/controllers/sitemap_controller.rb:

class SitemapController < ApplicationController
  layout nil

  def index
    headers['Content-Type'] = 'application/xml'
    last_post = Post.last
    if stale?(:etag => last_post, :last_modified => last_post.updated_at.utc)
      respond_to do |format|
        format.xml { @posts = Post.all }
      end
    end
  end
end

app/views/sitemap/index.xml.haml:

!!! XML
%urlset{:xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9"}
  - for post in @posts
    %url
      %loc #{post_url(post)}/
      %lastmod=post.updated_at.strftime('%Y-%m-%d')
      %changefreq monthly
      %priority 0.5

You can test it by bringing up localhost:3000/sitemap.xml.

Galbraith

For those who are getting ActionView::MissingTemplate error, simply add gem 'haml-rails to your gemfile and run bundle install. – Unfleshly

My question might sound very easy but I don't have any Post model... what is it ? – Monarchy

We were just using Post as an example model. You can name yours whatever you like. :-) – Galbraith

Answer 4

I would recommend that you check out the sitemap_generator gem. It handles all of these issues for you...and really, who wants to mess around authoring XML?

Here is an example sitemap to show how you use your Rails models and path helpers to generate your sitemap URLs:

# config/sitemap.rb
SitemapGenerator::Sitemap.default_host = "http://www.example.com"
SitemapGenerator::Sitemap.create do
  add '/contact_us'
  Content.find_each do |content|
    add content_path(content), :lastmod => content.updated_at
  end
end

Then you use Rake tasks to refresh as often as you would like. It really is that simple :)

Heine

Thanks for the tip, this gem really makes it easy to create the sitemap including pinging all major search engines. – Wilbanks

Answer 5

Here is a plugin for creating sitemaps in Ruby on Rails: Ruby on Rails sitemap plugin. It takes care of most of the sitemap logic and generation. The plugin is from my homepage.

Example configuration:

Sitemap::Map.draw do

  # default page size is 50.000 which is the specified maximum at http://sitemaps.org.
  per_page 10

  url root_url, :last_mod => DateTime.now, :change_freq => 'daily', :priority => 1

  new_page!

  Product.all.each do |product|
    url product_url(product), :last_mod => product.updated_at, :change_freq => 'monthly', :priority => 0.8
  end

  new_page!

  autogenerate  :products, :categories,
                :last_mod => :updated_at,
                :change_freq => 'monthly',
                :priority => 0.8

  new_page!

  autogenerate  :users,
                :last_mod => :updated_at,
                :change_freq => lambda { |user| user.very_active? ? 'weekly' : 'monthly' },
                :priority => 0.5

end

Best regards, Lasse

Mccahill

Answer 6

This article explains how a sitemap can be generated.

Basically should should create a controller which finds all pages (eg your Posts) and put in into an XML file. Next you tell Google about the location of the XML file and when your website is updated.

A simple Google rails sitemap query reveals lots of other articles explaining basically the same thing.

Batangas