Best practice to maintain a mgo session

Question

I'm currently using a mongodb with mgo lib for a web application, but I'm not sure if the way I'm using it, is good one ..

package db

import (
    "gopkg.in/mgo.v2"
)

const (
    MongoServerAddr = "192.168.0.104"
    RedisServerAddr = "192.168.0.104"
)

var (
    MongoSession, err = mgo.Dial(MongoServerAddr)

    MDB  = MongoSession.DB("message")
    MCol = MDB.C("new")
    MSav = MDB.C("save")

    UDB  = MongoSession.DB("account")
    UCol = UDB.C("user")
)

I init the db session and create variables who takes the collection and document value, so when I need to query a collection, I use the variable to make it.

Like that :

func UserExist(username string) bool {
    user := Users{}
    err := db.UCol.Find(bson.M{"username": username}).One(&user)
    if err != nil {
        return false
    } else {
        return true
    }
}

So is there a best practice or this one is fine ..? Thanks

Redbird

It's better practice to use a function for setting up the database session than variable declarations. One reason to use a function is that you can handle the error return from Dial. For UserExist, I would use the count of documents in the result set to determine if a document exists. There's no need to fetch the actual document. – Objectionable

thanks for the tip for the UserExist function! But with the function to init the session connection, can i do it with "func init()" in the db package and assign the global variable for db and collection with the return session ? I'm just not sure how to maintain my session with the db open, without making a "mgo.Dial()" every time i need it, and also have my db and collection already initialized ... – Redbird

Answer 1

I suggest not using a global session like that. Instead, you can create a type that is responsible for all the database interaction. For example:

type DataStore struct {
    session *mgo.Session
}

func (ds *DataStore) ucol() *mgo.Collection { ... }

func (ds *DataStore) UserExist(user string) bool { ... }

There are many benefits to that design. An important one is that it allows you to have multiple sessions in flight at the same time, so if you have an http handler, for example, you can create a local session that is backed by an independent session just for that one request:

func (s *WebSite) dataStore() *DataStore {
    return &DataStore{s.session.Copy()}
}    

func (s *WebSite) HandleRequest(...) {
    ds := s.dataStore()
    defer ds.Close()
    ...
}

The mgo driver behaves nicely in that case, as sessions are internally cached and reused/maintained. Each session will also be backed by an independent socket while in use, and may have independent settings configured, and will also have independent error handling. These are issues you'll eventually have to deal with if you're using a single global session.

Bakst

There are many reasons to have this setup like you stated. The session.Copy() call is especially important and something that isn't stressed enough to people using the Mongo driver in Go, given that it allows the driver to take full advantage of concurrency. But then - you know all about that .. :P +1 – Audubon

So you spin up an initial global session and then copy it on every request? – Cyndi

A master session, yes. It doesn't have to be global in the "global variable" sense of the word. – Bakst

Thank you. Is there issues with db consistency using Copy() over Clone()? – Cyndi

Quick question. What is the WebSite type you're using? And do you know of any tutorials that demonstrate this kind of approach? – Whimper

Let me get this straight: if I handle 5000reqs/sec I open up 5000 db sessions per second? Is that right? – Scrutator

@GustavoNiemeyer You are not explaining where the first session should be initialized and what the definition of WebSite is – Benzoyl

Does this work with multiple files? Say you have main.go where you set up the connection. You have a file in api/v1/users.go where you return a collection of users. How would you access the session from that users.go file? And as @kskyriacou pointed out, could you clarify all the fields you have in your code please – Vaquero

@codepushr no, it doesn't. mgo uses an internal pool of socket connections to mongodb. on each session.Copy(), it'll take one from there. – Callisto

This answer is not a minimal working example as others have pointed out. In the following post, a single mgo.Session is initialized in main() and is copied within a method run concurrently: mongodb.com/blog/post/… Concurrency in that example is simply looping over a method and is not asynchronous. Hence, it's not very useful for the scenario @codepushr mentioned, which requires queuing and async to be efficient. – Activist

Answer 2

Although not directly answering your question, regarding mgo session checking you must use defer/recover since mgo calls (even mgo.session.Ping) panic. As far as I can tell there is no other way of checking mgo session state (mgo godocs). You can use Gustavo Niemeyer's suggestion and add a method on your DataStore type.

func (d *DataStore) EnsureConnected() {
    defer func() {
        if r := recover(); r != nil {
            //Your reconnect logic here.
        }
    }()

    //Ping panics if session is closed. (see mgo.Session.Panic())  
    d.Ping()
}

Unwished

You can recover in your handlers with a recoverer middleware without this. – Callisto

@inanc How does the middleware accomplish this? DataStore.Ping? – Unwished

When it panicked, look at the error message and then reconnect. – Callisto

Do you have an example of a recoverer middleware? – Unwished

I wrote my own for MongoDb however there's a generic example here. – Callisto

Answer 3

With go 1.7, the most idiomatic way of handling mongo session on a webserver is to use the new standard library package context to write a middleware that can attach the defer session.Close() to whenever the request context Done() is called. So you do not need to remeber to close

AttachDeviceCollection = func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            db, err := infra.Cloner()
            if err != nil {
                http.Error(w, err.Error(), http.StatusInternalServerError)
                return
            }
            collection, err := NewDeviceCollection(db)

            if err != nil {
                db.Session.Close()
                http.Error(w, err.Error(), http.StatusInternalServerError)
                return
            }
            ctx := context.WithValue(r.Context(), DeviceRepoKey, collection)
            go func() {
                select {
                case <-ctx.Done():
                    collection.Session.Close()
                }
            }()

            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }

Undergraduate

Context are not meant to use for long lived things like mongo sessions. This is a smelly anti-pattern. – Callisto

@InancGumus Which method is the best way to handle mongo session? I'm seeing quite a few examples around using context and mongo sessions. – Factional

You can just copy the session from an injected struct member for example down the line. – Callisto

@InancGumus but isn't the context here providing a session for the short-lived handler? – Congressman