Node.js and MongoDB, reusing the DB object

Question

I'm new to both Node.js and MongoDB, but I've managed to put some parts together from SO and the documentation for mongo.

Mongo documentetion gives the example:

// Retrieve
var MongoClient = require('mongodb').MongoClient;

// Connect to the db
MongoClient.connect("mongodb://localhost:27017/exampleDb", function(err, db) {
  if(!err) {
    console.log("We are connected");
  }
});

Which looks fine if I only need to use the DB in one function at one place. Searching and reading on SO has shown me that I should not open a new connection each time, but rather use a pool and reuse the database object I get the first time. This answer is abundant on SO, but I'm not sure how to even get the DB object in the first place, and then how to reuse it.

Say I have the Node.js code above in my App.js, and I then have differnt routes that need to run different operations on the db like:

app.post('/employee', function(req, res){
    //Put req.name in database
});


app.post('/car', function(req, res){
    //Put req.car in database
});

How would I go about to put these two snippets together into something useful?

I found a similar question in Node.js reuse MongoDB reference , but from the looks of this ( http://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html ) it looks like I should use MongoClient rather than db(). And I'm not sure it solves my problem either...

Hackney

Answer 1

You could always write a module which initializes your database connections, and makes them accessible throughout your program. For example:

mongo.js

var mongodb = require('mongodb');

module.exports.init = function (callback) {
  var server = new mongodb.Server("127.0.0.1", 27017, {});
  new mongodb.Db('test', server, {w: 1}).open(function (error, client) {
    //export the client and maybe some collections as a shortcut
    module.exports.client = client;
    module.exports.myCollection = new mongodb.Collection(client, 'myCollection');
    callback(error);
  });
};

app.js

var mongo = require('./mongo.js');

//setup express...

//initialize the db connection
mongo.init(function (error) {
    if (error)
        throw error;

    app.listen(80); //database is initialized, ready to listen for connections
});

randomFile.js

var mongo = require('./mongo.js');

module.exports.doInsert = function () {
  //use the collection object exported by mongo.js
  mongo.myCollection.insert({test: 'obj'}, {safe:true}, function(err, objects) {
    if (err)
        console.warn(err.message);
  });
};

I know people talk about pooling, but when I did benchmarking of pooling mongo connections vs. a single connection for all requests, the single connection actually performed better. Granted, this was about a year ago, but I doubt that basic concept has changed. All the requests are asynchronous, so it's not like multiple connections are necessary in order to make simultaneous requests.

As far as MongoClient, I guess that's the new syntax they're encouraging. Either way, it's essentially a client object that you want to keep and make accessible regardless of which style you use.

Trivandrum

Hmm, will have to read up on modules and stuff before trying. "module.exports.init" looks wierd to me :) But what you're saying is that the code in "app.js" should only be run once, and then the code in "randomFile.js" is the one i use in my app.post('/employee')? Being my first time, I have everything in one App.js file. – Hackney

That's right, the init function should only be called once. The randomFile.js was just an example to show that you can use the initialized database from anywhere in the code. I could be used just as easily within app.js. You definitely should understand how modules work if you're going to be working with node, but they're really not difficult. – Trivandrum

Thanks a lot for your time and effort, I got it working today! When I start the server I get 5 connections to the DB instantly.. Is that bad? (Guessing 5 since it's the default allowed connections?) I did change the init call and how I use the module, see pastebin.com/QFdZY51F – Hackney

I'm not sure why you assume five connections. It should only be one – Trivandrum

Nono, I mean I do get 5 connetions everytime I turn on the node.js server (see log from turning it on, then off: pastebin.com/jGx54siH ). I think it's only 5 because I think I've seen that as some kind of default value for number of connections in mongo config. Might have been more otherwise I guess, but I'm not looping through any connection, so not really sure why it isn't only one. – Hackney

I never have because I'm constantly reusing the connection, and it's just pointless to close it as long as you aren't opening new connections. In theory, you should probably have graceful shutdown code which calls close before the process terminates. It's a good question, and maybe other people would have varying thoughts. – Trivandrum

I'll leave it open. Still don't get why it opens 5 connections when I start the server, any idea why this might be? – Hackney

I'm not sure, and I don't have an environment setup to readily test it myself at this moment. If you're really curious, I think it's a good fit for a separate stack overflow question . – Trivandrum

This is the single perfect answer for managing db connections with native node mongodb driver but somehow too difficult to find on Google. – Mckinnon

It looks like mongodb driver automatically does some connection pooling. That's why there are 5 connections and what MongoClient.connect() is doing is creating a connection pool. – Mckinnon

@BretCopeland How can I set myCollection from randomFile.js using mongo instance? – Stolon

@MuhammadShahzad I think maybe I don't understand your question. It could be assigned from anywhere via mongo.myCollection = .... I also don't have any idea if this answer still represents a best practice. I haven't used node.js in a couple years, and it's been even longer since I used Mongo. – Trivandrum

Answer 2

From your last code snippet, seems like you using Express or similar framework. You can use express-mongo-db to get connection inside req object. This middleware will cache connection for you and share it with other incoming requests:

app.use(require('express-mongo-db')(require('mongodb'), { db: 'test' }))
app.post('/employee', function(req, res){
    // req.db.collection('names'),insert(req.name)
    // Put req.name in database
});


app.post('/car', function(req, res){
    // req.db.collection('names').insert(req.name)
    // Put req.car in database
});

Gissing