MongoDB populate performance

Question

Recently asked such a question.

What is faster in Mongo: a populate or individual assync requests?

Example

var mongoose = require('mongoose')
    , Schema = mongoose.Schema;

var FeedPostCommentSchema = new Schema({

    feedPost: {type: Schema.Types.ObjectId, ref: 'FeedPost', index: true},
    user: {type: Schema.Types.ObjectId, ref: 'User'},

    comment: {
        type: String,
        trim: true
    },

    updated: {type: Date, default: Date.now},
    created: {type: Date, default: Date.now, index: true}
});

mongoose.model('FeedPostComment', FeedPostCommentSchema);

When displaying all the comments, we also need to get user data

We can make it standard populate method:

FeedPostComment.find({feedPost: req.params.postId}).populate('user')
        .skip(skip)
        .limit(limit)
        .sort({created: -1})
        .lean()
        .exec()

We can also do this with parallel queries:

FeedPostComment.find({feedPost: req.params.postId})
        .skip(skip)
        .limit(limit)
        .sort({created: -1})
        .lean()
        .exec(function (err, comments) {
            async.each(comments, function (comment, cb) {
                User.findById(comment.user, function (err, composer) {
                    if (err) {
                        return cb(err);
                    }

                    comment.user = utils.getPublicDataOfUser(composer);
                    cb();
                });
            }, function (err) {
                comments.reverse();
            });
        });

Ctesiphon

Answer 1

Populate would always be faster then normal async request... But now there is even faster than that has been introduced i.e. $lookup with pipeline... You should probably go with it.

Tactile

$lookup cannot be used with sharding so it limits horizontal scaling though? – Khaki

@Khaki From Mongo 5.1 $lookup works with sharding – Sniffy

Answer 2

Populate is generally faster (and more efficient) because it uses a single $in query to get all the referenced docs instead of getting them one by one.

Hurleigh