How can I promisify the MongoDB native Javascript driver using bluebird?

Question

I'd like to use the MongoDB native JS driver with bluebird promises. How can I use Promise.promisifyAll() on this library?

Unbidden

mongo 2.0 onwards lets you tell the mongo driver that you want to use bluebird: easyusedev.wordpress.com/2015/10/07/… – Daves

This in my opinion is the best solution, would be great for you to put it in as an answer and not a comment. – Moen

Answer 1

The 2.0 branch documentation contains a better promisification guide https://github.com/petkaantonov/bluebird/blob/master/API.md#promisification

It actually has mongodb example which is much simpler:

var Promise = require("bluebird");
var MongoDB = require("mongodb");
Promise.promisifyAll(MongoDB);

Incorruption

@dimadima you need to promisify Cursor.prototype (which the above code does) not Cursor – Incorruption

That doesn't work because of github.com/mongodb/node-mongodb-native/blob/1.4/lib/mongodb/…; you see the cursor that's returned is not built from Cursor.prototype. – Unbidden

@dimadima I see, that is incredibly inefficient to create the methods over and over again for each cursor :/ – Incorruption

yes, it's not great. That's part of the reason I posted this Q/A: the first part of my answer is obvious if you guess that normal promisification procedure is in order; but getting to figure out what's happening with the cursor takes some stepping through code if reading it is not immediately obviously. – Unbidden

Isn't this just var MongoDB = Promise.promisifyAll(require("mongodb")) now? – Amperehour

Correct, now you only need to promisify require("mongodb"). All cursors then have toArrayAsync method, for example. – Wolgast

I have just updated from an old version of mongo 1.4.* to the newer 2.* versions and currently this approach which used to work no longer seems to work, the MongoClient object seems to be missing the connectAsync method etc. – Sevastopol

@Incorruption docco has moved to bluebirdjs.com/docs/api/promise.promisifyall.html with 3.0. No mongodb example any more – Sap

Answer 2

When using Promise.promisifyAll(), it helps to identify a target prototype if your target object must be instantiated. In case of the MongoDB JS driver, the standard pattern is:

• Get a Db object, using either MongoClient static method or the Db constructor
• Call Db#collection() to get a Collection object.

So, borrowing from https://mcmap.net/q/538287/-mongodb-promise-gets-returned-too-early-closed, you can:

var Promise = require('bluebird');
var mongodb = require('mongodb');
var MongoClient = mongodb.MongoClient;
var Collection = mongodb.Collection;

Promise.promisifyAll(Collection.prototype);
Promise.promisifyAll(MongoClient);

Now you can:

var client = MongoClient.connectAsync('mongodb://localhost:27017/test')
    .then(function(db) {
        return db.collection("myCollection").findOneAsync({ id: 'someId' })
    })
    .then(function(item) {
      // Use `item`
    })
    .catch(function(err) {
        // An error occurred
    });

This gets you pretty far, except it'll also help to make sure the Cursor objects returned by Collection#find() are also promisified. In the MongoDB JS driver, the cursor returned by Collection#find() is not built from a prototype. So, you can wrap the method and promisify the cursor each time. This isn't necessary if you don't use cursors, or don't want to incur the overhead. Here's one approach:

Collection.prototype._find = Collection.prototype.find;
Collection.prototype.find = function() {
    var cursor = this._find.apply(this, arguments);
    cursor.toArrayAsync = Promise.promisify(cursor.toArray, cursor);
    cursor.countAsync = Promise.promisify(cursor.count, cursor);
    return cursor;
}

Unbidden

calling promisify on cursor.toArray is expensive if you do it each time, what about promisifying Cursor.prototype ? – Amperehour

Also, aren't you missing the Async prefix in the "Now you can" part? – Amperehour

@BenjaminGruenbaum: Yup, realized I forged the -Async after I closed the tab out. Just fixed that. Got the impression from the referenced answer that the cursor isn't built from a prototype. Testing that out now. – Unbidden

Also, you probably want to promisify MongoClient.prototype instead of MongoClient too. – Amperehour

It's not magic, you know :) github.com/mongodb/node-mongodb-native/blob/1.4/lib/mongodb/… – Amperehour

@BenjaminGruenbaum: Yes, you could do that too if you wanted to use any of the non-static methods on MongoClient. – Unbidden

@BenjaminGruenbaum totally. Nothing better than demystifying things :D. Will test locally regarding the Cursor construction because of the aforementioned referenced answer. – Unbidden

@BenjaminGruenbaum: So, indeed, while Cursor has a prototype, that prototype isn't used in constructing the Cursor that's ultimately passed by Collection#find(). Have a look at github.com/mongodb/node-mongodb-native/blob/1.4/lib/mongodb/…, which is called via github.com/mongodb/node-mongodb-native/blob/1.4/lib/mongodb/…. All of that said, I don't think Promisification is all that expensive, relative to, you know, disk and network IO. – Unbidden

@dimadima node is bound by CPU, if processing stuff didn't take CPU node could handle infinite IO – Incorruption

@Esailija. Very interesting. Given my knowledge of Node, I wouldn't think that's the case. Just because it's not blocking, doesn't mean it's not waiting to be called back in the loop/queue? Obviously, though, you know way more about this than me—could you please point me to something to read on this topic? Thank you for bluebird. – Unbidden

@dimadima sure the individual request won't come up any faster, but the less you use CPU the more concurrent independent requests you can serve, theoretically up to infinity. – Incorruption

@Esailija: Yes, now that does make sense, of course. Simple. Thank you. – Unbidden

@BenjaminGruenbaum, why...? His answer is incomplete. Do you actually read things or just spaz around? You never replied to my comment about promisifying the cursor. "It's not magic", and then you disappeared. Thanks for nothing on this thread, except being unpleasant and unhelpful. Why do you even bother? – Unbidden

I'm sorry you fill discontent, I just want people to see Esailija's answer first when they answer this question - if you check out Bluebird recently exposed functionality that makes this a one liner. Bluebird will now find and promisify the cursor automatically on its own. If you accept his answer, it'll be first anyway, and I'll gladly remove this downvote. – Amperehour

@BenjaminGruenbaum but that is incorrect. Bluebird is not able to promisify the cursor automatically on its own. That's the part about reading. You didn't read my explanation, nor did you test this out on your own. So you are incorrect. Do you understand that? If you were sorry, or really, more importantly, sincere to any degree, you would take a minute, like I did, to verify the behavior. – Unbidden

I did verify it, today, which is why I left that comment on the other answer. Clone the 2.0 branch and try it for yourself. – Amperehour

@BenjaminGruenbaum, well, I will do that. I thought he was referring to the documentation. I apologize, and understand, if what you're saying is correct. Which must be the case. – Unbidden

You can be glad, questions like this one are what changed the behavior in 2.0, it will now look for functions aggressively, and promisify them, we're also adding a guide on how to promisify common libraries and we can use all the help we can get :) – Amperehour

Answer 3

I know this has been answered several times, but I wanted to add in a little more information regarding this topic. Per Bluebird's own documentation, you should use the 'using' for cleaning up connections and prevent memory leaks. Resource Management in Bluebird

I looked all over the place for how to do this correctly and information was scarce so I thought I'd share what I found after much trial and error. The data I used below (restaurants) came from the MongoDB sample data. You can get that here: MongoDB Import Data

// Using dotenv for environment / connection information
require('dotenv').load();
var Promise = require('bluebird'),
    mongodb = Promise.promisifyAll(require('mongodb'))
    using = Promise.using;

function getConnectionAsync(){
    // process.env.MongoDbUrl stored in my .env file using the require above
    return mongodb.MongoClient.connectAsync(process.env.MongoDbUrl)
        // .disposer is what handles cleaning up the connection
        .disposer(function(connection){
            connection.close();
        });
}

// The two methods below retrieve the same data and output the same data
// but the difference is the first one does as much as it can asynchronously
// while the 2nd one uses the blocking versions of each
// NOTE: using limitAsync seems to go away to never-never land and never come back!

// Everything is done asynchronously here with promises
using(
    getConnectionAsync(),
    function(connection) {
        // Because we used promisifyAll(), most (if not all) of the
        // methods in what was promisified now have an Async sibling
        // collection : collectionAsync
        // find : findAsync
        // etc.
        return connection.collectionAsync('restaurants')
            .then(function(collection){
                return collection.findAsync()
            })
            .then(function(data){
                return data.limit(10).toArrayAsync();
            });
    }
// Before this ".then" is called, the using statement will now call the
// .dispose() that was set up in the getConnectionAsync method
).then(
    function(data){
        console.log("end data", data);
    }
);

// Here, only the connection is asynchronous - the rest are blocking processes
using(
    getConnectionAsync(),
    function(connection) {
        // Here because I'm not using any of the Async functions, these should
        // all be blocking requests unlike the promisified versions above
        return connection.collection('restaurants').find().limit(10).toArray();
    }
).then(
    function(data){
        console.log("end data", data);
    }
);

I hope this helps someone else out who wanted to do things by the Bluebird book.

Plata

Answer 4

Version 1.4.9 of mongodb should now be easily promisifiable as such:

Promise.promisifyAll(mongo.Cursor.prototype);

See https://github.com/mongodb/node-mongodb-native/pull/1201 for more details.

Typescript

Answer 5

We have been using the following driver in production for a while now. Its essentially a promise wrapper over the native node.js driver. It also adds some additional helper functions.

poseidon-mongo - https://github.com/playlyfe/poseidon-mongo

Sarchet