how to generate a sitemap in expressjs
Asked Answered
U

6

12

I downloaded my XML sitemap from the sitemap xml generator website. I placed my sitemap.xml on my public directory but when I tried to submit the sitemap.xml into google console i received the following error: General HTTP error: 404 not found HTTP Error: 404
So i coded
app.get('/sitemap.xml', function( req, res, next ) { res.header('Content-Type', 'text/xml'); res.render( 'sitemap' ); )};

And when i navigate to the 'website/sitemap.xml' I am getting the following error:

This page contains the following errors:

error on line 1 at column 42: Specification mandate value for attribute itemscope

Thanks for your help

Unpredictable answered 12/5, 2016 at 18:18 Comment(4)
Why not use express-sitemap to generate it dynamically rather than laboriously copying and pasting it? If you're using a proper web server like Nginx, it should be straightforward to serve it with that.Ringdove
your header should be res.header('Content-Type', 'application/xml');Imco
Use this: github.com/ekalinin/sitemap.jsPossessed
Make sure app.js has app.use(express.static(path.join(__dirname, 'public')));Reparation
J
11

Generate your sitemap.xml file using a tool like https://www.xml-sitemaps.com/

upload the sitemap.xml in your project

then add this to your .js file:

router.get('/sitemap.xml', function(req, res) {
res.sendFile('YOUR_PATH/sitemap.xml');
});

make sure you change YOUR_PATH for the actual path where your sitemap.xml file is.

Jag answered 4/10, 2019 at 19:0 Comment(1)
for me only work like this: res.sendFile('sitemap.xml', { root: '.' })Team
S
5

Sitemaps do not have to be XML documents. A simple text file with URLs is all you need so something like below works fine. In the following example, fetchMyUrls() would be a function/method that asynchronously gets and returns the available URLs as an array of strings (URL strings).

async function index (req, res){
    return fetchMyUrls().then((urls) => {
      var str = '';
      for (var url of urls) {
        str = str + url + '\n';
      }
      res.type('text/plain');
      return res.send(str);
    });  
}
Swedish answered 20/1, 2019 at 19:19 Comment(0)
E
5

For those looking for a way to create the XML dynamically on your code and don't want to use another library nor have a file stored in the public folder, you can use this:

app.get('/sitemap.xml', async function(req, res, next){
  let xml_content = [
    '<?xml version="1.0" encoding="UTF-8"?>',
    '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">',
    '  <url>',
    '    <loc>http://www.example.com/</loc>',
    '    <lastmod>2005-01-01</lastmod>',
    '  </url>',
    '</urlset>'
  ]
  res.set('Content-Type', 'text/xml')
  res.send(xml_content.join('\n'))
})
Emerick answered 8/4, 2020 at 23:46 Comment(4)
does next() need to be called at the bottom of this function? Edit: And, just curious, could this approach also be used for generating and serving /ads.txt and /robots.txt content - i am getting 404 errors on these three routes in my logs even though my app isn't calling them specifically.Concordia
I think it depends on how you are handling 404 (or other errors), if your app is catching first the error and is not reaching the point where you make the calling of /robots.txt then it will not work. On my app, the error catching takes places at the very end. Just before app.listen().Islam
And yes, I use this same approach for generating /ads.txt and /robots.txt. Only instead of using res.set('Content-Type', 'text/xml') for txt file you will need res.type('text/plain')Islam
the next() is only required if you are using a middleware (which I assume you are not using).. Here the official Doc: Writing middleware for use in Express appsIslam
F
1

In my NodeJS express project and without installing any library I was able to add this to my routes with my preferred view engine (handlebar).

export const routes: RouteMapper[] = [
    {
        "/sitemap.xml": [
        {
            method: "get",
            handler: (req, res) =>
            res.sendFile("/src/views/sitemap.xml", { root: "." }),
        },
      ],
    },
];

Cheers!

Finkle answered 13/11, 2020 at 16:42 Comment(0)
D
1

The best way is to create a script that would automatically generate a sitemap. In a lot of cases, the URLs should be dynamic based on data from the database.

Great package for creating the sitemap in Express is sitemap package:

STEP 1

Create a middleware that will generate the sitemap dynamically and then cache it for each next call to the server. We can extract logic in separate file called sitemap_generator.js for example, and we can define and export generate_sitemap middleware for it:

const { SitemapStream, streamToPromise } = require('sitemap');
const { Readable } = require('stream');
let sitemap;

const generate_sitemap = async (req, res, next) => {

  res.header('Content-Type', 'application/xml');

  if (sitemap) return res.status(200).send(sitemap); // If we have a cached entry send it

  let changefreq = 'weekly';

  try {

    let links = [
      { url: '', changefreq, priority: 1 },
      { url: 'aboutus', changefreq, priority: 0.9 },
      { url: 'blog', changefreq },
      { url: 'login', changefreq },
      { url: 'register', changefreq },
    ];

    // Additionally, you can do database query and add more dynamic URLs to the "links" array.

    const stream = new SitemapStream({ hostname: 'https://example.com', lastmodDateOnly: true })
    return streamToPromise(Readable.from(links).pipe(stream)).then((data) => {
      sitemap = data; // Cache the generated sitemap
      stream.end();
      return res.status(200).send(data.toString())
    });

  } catch (error) {
    return res.status(500).end();
  }
}

module.exports = { generate_sitemap };

STEP 2

Import generate_sitemap middleware from sitemap_generator.js in your server configuration file and mound it to the /sitemap.xml endpoint:

const { generate_sitemap } = require('./sitemap_generator');

...

app.get('/sitemap.xml', generate_sitemap);

That's it. Your sitemap should be available on /sitemap.xml endpoint now so navigate in the browser to that endpoint and check if it is there.

Diageotropism answered 26/8, 2021 at 2:34 Comment(0)
I
1

After generating a sitemap.xml file from a website - take https://www.xml-sitemaps.com/ for example - you can simply copy/paste:

app.use('/sitemap.xml', function (req, res, next) {
      res.type('text/xml')
      res.send(
  `<?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
        <loc>https://yourwebsite.com/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>weekly</changefreq>
        <priority>1.0</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page1/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.9</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page2/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.7</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page3/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>yearly</changefreq>
        <priority>0.6</priority>
     </url>
     <url>
        <loc>https://yourwebsite.com/page5/</loc>
        <lastmod>2023-03-28T00:00:00+00:00</lastmod>
        <changefreq>yearly</changefreq>
        <priority>0.5</priority>
     </url>
  </urlset>`);
  });
Impart answered 31/3, 2023 at 11:26 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.