Dynamic sitemap in ASP.NET MVC

Question

I'm trying to create an automatic sitemap ActionResult that outputs a valid sitemap.xml file. The actual generation of the file is not a problem, but I can't seem to figure out how to populate the list of URL's in the system. Here is the code I have so far:

    public ContentResult Sitemap()
    {
        XNamespace xmlns = "http://www.sitemaps.org/schemas/sitemap/0.9";
        XElement root = new XElement(xmlns + "urlset");

        //some kind of foreach here to get the loc variable for all URLs in the site
        //for each URL in the collection, add it to the root element as here

        //root.Add(
        //    new XElement("url", 
        //        new XElement("loc", "http://google.com"), 
        //        new XElement("changefreq", "daily")));

        using (MemoryStream ms = new MemoryStream())
        {
            using (StreamWriter writer = new StreamWriter(ms, Encoding.UTF8))
            {
                root.Save(writer);
            }

            return Content(Encoding.UTF8.GetString(ms.ToArray()), "text/xml", Encoding.UTF8);
        }
    }

For instance, suppose I have two controllers, and each controller has two actions associated with them:

HelpController

• Edit
• Create

AboutController

• Company
• Management

I can't seem to figure out how to get a list of URL's like:

• http://localhost/help/edit
• http://localhost/help/create
• http://localhost/about/company
• http://localhost/about/management

Laterality

Recently, like answer from @eduncan911, the best solution is to use mvcsitemap.codeplex.com active and updated project, support for security trimming and will generate sitemap.xml. It also can automatically compress the sitemap if agent support for it and split the sitemap into sub-sitemaps if the site is too large because sitemap.xml standard have the limit to 50k nodes only. – Magnetometer

Thanks CallMeLaNN. I recently updated the answer to list those points and more, as well as listing the site it moved to. – Superstition

Answer 1

I took a look at Maarten Balliauw's approach per likwid's comment, but it seems to be overkill for what I'm trying to do.

I've hacked together a temporary solution. I'm simply passing the controller and action names to generate the URL's. In order to generate the URL's, I'm using the following code:

    List<string> urlList = new List<string>();
    urlList.Add(GetUrl(new { controller = "Help", action = "Edit" }));
    urlList.Add(GetUrl(new { controller = "Help", action = "Create" }));
    urlList.Add(GetUrl(new { controller = "About", action = "Company" }));
    urlList.Add(GetUrl(new { controller = "About", action = "Management" }));

where GetUrl is as below:

    protected string GetUrl(object routeValues)
    {
        RouteValueDictionary values = new RouteValueDictionary(routeValues);
        RequestContext context = new RequestContext(HttpContext, RouteData);

        string url = RouteTable.Routes.GetVirtualPath(context, values).VirtualPath;

        return new Uri(Request.Url, url).AbsoluteUri;
    }

This seems to do the trick for now, though I do like the idea of having actionfilter's applied to certain actions that get pulled together automatically.

Laterality

seems good ,,but if i need to add a paremeter also..like /About/Management?level='top' how can that be done ..is it using id in routevalue – Trishatriskelion

Answer 2

I posted a do-it-yourself answer down below. But here is a package that does it out of the box for MVC sites:

http://mvcsitemap.codeplex.com/ (<- old site, but with extensive documentation!)

https://github.com/maartenba/MvcSiteMapProvider/wiki (<- moved to new site, lacking some documentation, and not as active)

Note that it does a multitude of things:

• Automagically registers itself in the Mvc routes to respond to SEO /sitemap.xml requests (even though there is no physical file for /sitemap.xml). This is completely compatible with all search engine robots I've found, as well as rolling over when it gets to 10,000, etc.
• Comes with a set of partial views to use for BreadCrumb navigation built-in! We use this quite extensively, though the dynamic data portion is a bit cumbersome, it does work.
• Comes with a set of partial views for Menu to be controlled as well.
• Honors the [Authorize] security bits of your Controllers and Action methods.

All of the above points are controlled from the single mvc.sitemap XML file you edit and configure. I've used this in a number of projects now to do 2 or 3 of the above points. Have it all configurable in 1 place, and dynamically generated, is really nice.

Though I find the ability to create dynamic data providers a bit cumbersome (and grossly violates any type of IoC you wish to do), it does get the job done and scales nicely once you bypass their caching and use your own.

Superstition

That's an ASP.NET sitemap not a sitemap.xml file for a search engine – Yoheaveho

@Hightechrider it creates a sitemap.xml for your registered maps. Enabling is trivial. github.com/maartenba/MvcSiteMapProvider/wiki/… – Lemmons

Unfortuantly the project owner of MvcSiteMap has moved off of CodePlex, which had a thriving community around MvcSitemap, and moved to GitHub, where it is a Ghosttown for this project (we've reported a few bugs, zero response from the "single" person authoring it). Not to mention how well documented the CodePlex project is, and how the Github version is lacking everything. Boo. – Superstition

^- Coming back to comment on my comment above. It looks like interest has picked up the Ghost town has been re-populated with people actively contributing (about time!). Though, the issues list indicate some spotty unstable releases as of this date of the comment. I am updating this answer to point to the new Github site. – Superstition

To respond to Ian Mercer, you are incorrect. This is a provider that generates an /Sitemap.xml for MVC websites. Anyone can view the link and see the details. Hence, the name, "MVC Sitemap Provider" – Superstition

Answer 3

As likwid mentions, you want to reflect upon your model(s) namespace and obtain all classes that implement IController. Once you have the collection, you want to reflect to see what Members (methods) return the type ActionResult.

Perhaps you can create your own attribute, [SitemapAttribute] that lets you selectively specify what methods to index in the sitemap (i.e., Index(), but not Edit()). Yeah, I like that idea of controlling which methods (urls) gets written.

This is an excellent question because I was just thinking of doing the same. +1!

// Controller abstract implements IController
public class HelpController : Controller
{
  public HelpController()
  {
  }

  [Sitemap]
  public ActionResult Index()
  {
    // does get written to the file, cause of [Sitemap]
  }

  public ActionResult Create()
  {
    // does not get mapped to the file
  }

  public ActionResult Edit()
  {
    // does not get mapped to the file
  }

  [Sitemap]
  public ActionResult ViewArticle()
  {
    // would get indexed.
  }
}

For how to do reflection, here's a good MSDN article to get you introduced to reflection:

http://msdn.microsoft.com/en-us/library/ms172331.aspx

Good question!

Superstition

Can you post an example of using reflection to build the content? Specifically how to get the URL from the action? – Iorio

THanks Mike, but I do not have any code. It was just a suggestion. Below seems to be other examples. – Superstition

Answer 4

Define an ActionFilterAttribute like this to put on any Action method that is an actual page that you want to list in your sitemap:-

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = false)]
public class MVCUrlAttribute : ActionFilterAttribute
{
    public string Url { get; private set; }

    public MVCUrlAttribute(string url)
    {
        this.Url = url;
    }

    public override void OnResultExecuting(ResultExecutingContext filterContext)
    {
        // Put this 'canonical url' into the model (which feeds the view)
        // to help search engines with issues of duplicate content
        filterContext.Controller.ViewData["CanonicalUrl"] = url;
        base.OnResultExecuting(filterContext);
    }
}

Now add something like this to your Global application start code, or use it in your sitemap.xml generating code:-

   // Find all the MVC Routes
    Log.Debug("*** FINDING ALL MVC ROUTES MARKED FOR INCLUSION IN SITEMAP");
    var allControllers = Assembly.GetExecutingAssembly().GetTypes().Where(t => t.IsSubclassOf(typeof(Controller)));
    Log.DebugFormat("Found {0} controllers", allControllers.Count());

    foreach (var controllerType in allControllers)
    {
        var allPublicMethodsOnController = controllerType.GetMethods(BindingFlags.Public | BindingFlags.Instance);
        Log.DebugFormat("Found {0} public methods on {1}", allPublicMethodsOnController.Count(), controllerType.Name);

        foreach (var publicMethod in allPublicMethodsOnController)
        {
            var mvcurlattr = publicMethod.GetCustomAttributes(true).OfType<MVCUrlAttribute>().FirstOrDefault();
            if (mvcurlattr != null)
            {
                string url = mvcurlattr.Url;
                Log.Debug("Found " + controllerType.Name + "." + publicMethod.Name + " <-- " + url);
                Global.SiteMapUrls.Add(url);  //<-- your code here using url
            }
        }
    }

You can extend the attribute class to perhaps also include the frequency of update hint.

Yoheaveho

Hi. Is this sitemap valid according to google's guidelines? – Sleekit

Answer 5

So, the getting of the controllers and actions seems to me to be the relatively trivial part. The hard part is being able to get all the possible parameter values that you might want to show in the urls of your sitemap. If you have a URL pattern like {controller}/{action}/{id}, then you're not going to be able to determine through reflection what the meaning of id is, or the possible values. The best you can do is determine the system type.

What occurred to me as I was looking at this is that a sitemap is really just another view of your site's data. So one random thought I had was that if you inherit from a base controller in your app, and you have a method on that base controller that has to be implemented, e.g.:

abstract ActionResult SiteMapSnippet();

Then you could create a SiteMapController which calls each of the other controllers in the solution and asks them for their snippet, and then renders them all together in one final view. Sort of a composite controller, though that's not a concept that's been added to this framework yet.

Sialagogue