sitemap not created until server restart in meteor
Asked Answered
D

2

8

I am using meteor to create simple blog system. For sitemaps files I'm using this package.

I added some initialize data in server startup function (create some post) and used below code (server/sitemaps.js) in server to create sitemaps for each category (e.g. sitemap1.xml for first category and etc):

function sitemapOutput(categoryName){
    var out = [], posts = Posts.find({ category: categoryName }).fetch();
    _.each(posts, function(post) {
        out.push({
            page: post.url(),
            lastmod: post.insertDate,
            changefreq: 'weekly'
        });
    });
    return out;
}

Categories.find().forEach(function(Category, index) {
    sitemaps.add('/sitemap' + (index+1) +'.xml',
        function(){ return sitemapOutput(Category.name); });
});

And I have startup like this: (server/startup.js)

Meteor.startup(function () {
    // some post and category created here
});

But sitemaps didn't exist until server restart (my robots.txt files also empty) but when server restarted sitemaps and robots.txt content created for me.

I think posts inserted after sitemaps.js but what's the problem guys and how to fix that?

New try:

I try new solution like below but this code also didn't work. (I want to create seperate sitemap file for each 10000 category to prevent big sitemap and google sitemap error):

for (var i=0;i<=Math.round(Categories.find().count()/10000);i++) {
    sitemaps.add('/sitemap' + i +'.xml', function(){
        var out = [];
        Categories.find({}, {sort: {insertDate: 1} ,limit: 10000, skip: i * 10000}).forEach(function(Category) {
            out.push({
                page: "/category/" + Category.title + "/" + Category._id,
                lastmod: Category.insertDate,
                changefreq: 'weekly'
            });
        });
        return out;
    });
}

robots.txt show sitemap files correctly but all sitemap is empty like this:

<urlset> </urlset>

When does sitemaps.add() run? I think it does on server restart but New try was disappointed me and I think my guess is incorrect and If sitemaps.add() was run why it's empty.

Decorous answered 14/8, 2015 at 6:43 Comment(2)
What is your folder structure like?Lytton
I described in question. I have a sitemaps.js in server folderDecorous
M
3

Your problem seems to be the folder structure. You said you have /server/sitemaps.js and /server/startup.js and you wish that sitemaps would run after your startup, but the thing is Meteor will run those files alphabetically, so sitemap comes before startup. If you place your startup.js inside a lib folder, such as /server/lib/startup.js, you'll get the desired results, because Meteor will run lib folder before others.

Malaspina answered 1/9, 2015 at 13:28 Comment(9)
I think alphabetically order not for here because sitemaps.js code run as route when visit /sitemap.xml not in sequence order. What's your idea about second code?Decorous
You didn't say you have a route for displaying your sitemaps. Can you post the route code here? Also, you said that the second code gives you correct results in robots.txt, but what is the code generating robots.txt? Does it have all the sitemap1.xml, sitemap2.xml, etc., correctly?Malaspina
I said that second code worked as route. Yes robots.txt contains all sitemaps (sitemap1.xml, sitemap2.xml,...) but all files empty.Decorous
I haven't route. sitemaps package author said in this issue (github.com/gadicc/meteor-sitemaps/issues/13) that sitemaps.add() function create sitemap on the fly (for example when request to sitemap1.xml received) this behaviour very close to route functionality. For second code I only have a server/sitemaps.js file that generate sequence sitemaps files but all files are empty.Decorous
That's your problem then. See, when this file is loaded by the server, it's loaded before your /server/startup.js and once it runs, it generates all those empty sitemap.xml files. To see if it would make a difference, after you run your meteor instance, try to run meteor shell on another terminal and copy and paste the exact same code you had on your second attempt.Malaspina
Your answer is correct for first sitemaps. However unfortunately second sitemap still not working and generates empty sitemaps files. Although I think second code issue is different than the first and because of that I close this question but If you found second code answer I'm being happy to know issue. Thanks for everything man.Decorous
It's so interesting that second code generate empty sitemaps even after server restart. Now I say second code has deference issue with confidence. Maybe it's better to edit this question and move second code to new question. What's your idea?Decorous
You should probably open a new question for that, and maybe reference this one there. I don't see any problem with your code. Since this code is running on server, I don't think your count would be a problem. And you said the files are being created, but they are all empty, right? So it's not a problem with the category count, but with the code that runs inside the sitemap.add method.Malaspina
You should try using var categories = Categories.find({}, {sort: {insertDate: 1} ,limit: 10000, skip: i * 10000}).fetch() and console.log(categories.length) just to check that 10.000 categories are being returned by your find. Then you can use the _.each as you did in your first attempt.Malaspina
N
1

It's normal behavior, the code at Meteor.startup will run just once at app start. If you're looking to re run this function you either need to use meteor method to call the function from the client or you can you use something like cron job to run repeat jobs here is a great package https://atmospherejs.com/percolate/synced-cron

Nadaha answered 14/8, 2015 at 20:25 Comment(1)
I don't want to re run my startup code. My startup is some initialize data but didn't display in sitemaps.Decorous

© 2022 - 2024 — McMap. All rights reserved.