How does mongoose populate work under the hood

Question

Could somebody tell me how

I have a collection

a {
 b: String
 c: Date
 d: ObjectId --> j
}

j {
 k: String
 l: String
 m: String
}

when I carry out a:

a.find({ b: 'thing' }).populate('d').exec(etc..)

in the background is this actually carrying out two queries against the MongoDB in order to return all the items 'j'?

I have no issues getting populate to work, what concerns me is the performance implications of the task.

Thanks

Knudsen

Answer 1

Mongoose uses two queries to fulfill the request.

The a collection is queried to get the docs that match the main query, and then the j collection is queried to populate the d field in the docs.

You can see the queries Mongoose is using by enabling debug output:

mongoose.set('debug', true);

Gainor

@Gainor are both queries handled by the mongoose code in the server or is this tuned to make the population work on the DB and return the complete data? I'm worried this will result in (at least) two roundtrips to the DB... I know this is a helper function for "cleaner" code and it is very useful indeed, but in some use cases I'm considering optimizing the way the queries are executed to avoid multiple linear (sync) queries and make two (or more) parallel queries instead :-) – Pavilion

@Pavilion Mongoose uses two queries, resulting in two round trips. You can see the queries it's using by enabling debug output: mongoose.set('debug', true); – Gainor

Thanks @JohnnyHK, I didn't know about the debug mode to analyze mongoose's activity. Will definitely keep this in mind when defining my schemas. – Pavilion

Answer 2

Adding to @JohnnyHK answer on the performance implications of the task you worried about, I believe no matter what, these queries have to execute sequentially whether we use the mongoose provided populate() method or the one you will implement the server-side, both will have the same time complexity.

This is because in order to populate we need to have the results from the first query, after getting the result uuid will be used to query the document in the other collection.

So I believe it's a waste to make these changes the server-side than to use the mongoose provided method. The performance will remain the same.

Crackling

true. If one query was not dependent on another query's output, then it would be possible to run two seperate queries using two different db connections in order to get the documents concurrently. This would lead to better performance especially if first query is time consuming and second one is not. – Fenella

Answer 3

Basically the model 'a' is containing an attribute 'd' which is referencing(pointing) towards the model 'j'.

So whenever we use

a.find({ b: 'thing' }).populate('d').exec(etc..)

Then through populate we can individually call properties of 'j' like :

• d.k
• d.l
• d.m

Populate() helps us to call properties of other models.

Dongdonga

The question is whether "two queries against the MongoDB" are carried out or not. Maybe you can clarify it? – Impuissant