Should I copy session for each operation in mgo?

Question

I want to upsert a list of record, so I have two choice, one just use one session, another copy a session for every record. So, as my opinion, first method may slower than the second, but will the first one cause too many session created?

1.use one session

func (this *CvStoreServiceImpl) SetCvJobItemMeasureList(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItemList []*cv_common_type.CvJobItemMeasure) (err error) {
    session := this.session.Clone()
    defer session.Close()

    for _, jobItem := range jobItemList {
        objKey := &orm.ItemIdKey{
            VersionName: versionPolicy, //XXX
            ItemId:      jobItem.ItemId,
        }
        obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
        _, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
        if nil != err2 {
            err = &common_error.NamedError{err2.Error()}
            this.logger.Println(err2.Error())
        }
    }
    return
}

2.copy session for every record

func (this *CvStoreServiceImpl) SetCvJobItemMeasure(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItem *cv_common_type.CvJobItemMeasure) (err error) {
    session := this.session.Clone()
    defer session.Close()

    objKey := &orm.ItemIdKey{
        VersionName: versionPolicy, //XXX
        ItemId:      jobItem.ItemId,
    }
    obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
    _, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
    if nil != err2 {
        err = &common_error.NamedError{err2.Error()}
        return
    }
    return
}

then call this method in forloop:

for _, item := range cvMeasure.GetJobList() {
    err = this.SetCvJobItemMeasure(accessToken, versionPolicy, item)
    if nil != err {
        return
    }
}

Eris

I posted the wrong link here previously. Here's the right one: #26575094 – Ramin

Answer 1

First of all, we need to see the difference between mgo.Session.Copy() and mgo.Session.Clone(). While go.Session.Clone() returns a new session, the session uses the same socket connection. That isn't necessarily a bad thing, but keep in mind that on the server side, a stack is allocated per connection. So the sessions would share the same stack. Depending on your use cases, that may make a big difference.

And here is the problem – if you open a new socket connect for each record, this leads to a three way handshake, which is slowish. Reusing the same socket reduces this overhead, but there still is some and has the drawback described above.

What I tend to do is to establish a new connection per long(er) running unit of work. A simple example illustrates this:

package main

import (
    "fmt"
    mgo "gopkg.in/mgo.v2"
    bson "gopkg.in/mgo.v2/bson"
    "net/http"
)

var (
    Database *mgo.Database
)


// The listEntries lists all posts
func listPosts(w http.ResponseWriter, r *http.Request) {

    // We have a rather long running unit of work
    // (reading and listing all posts)
    // So it is worth copying the session   
    collection := Database.C("posts").With( Database.Session.Copy() )

    post  := bson.D{}
    posts := collection.Find(bson.M{}).Iter()

    for posts.Next(&post) {
        // Process posts and send it to w
    }

}

func main() {

    session, _ := mgo.Dial("mongodb://localhost:27017")

    Database := session.DB("myDb")

    // Count is a rather fast operation
    // No need to copy the session here
    count, _ := Database.C( "posts" ).Count()

    fmt.Printf("Currently %d posts in the database", count )

    http.HandleFunc("/posts", listPosts)
    http.ListenAndServe(":8080", nil)
}

Hound

I like this approach. However, how do you deal with closing sessions? I saw people use defer to close sessions at some points in time. Is this really necessary? If so, how should we incorporate this into your example? – Gasolier

@codepushr Uuups... ;) Feel free, or I'll do it later – Hound

I'm not sure how it's done properly though, probably defer session.Close() inside main, and also in the handler but using the copy? – Gasolier

Answer 2

yes, it is good to copy a session to perform one or several operations, letting the connection pool in mgo to improve the performance. the default limit is 4096 for one mongo server, to prevent too much connection.

func newSession(consistency Mode, cluster *mongoCluster, timeout time.Duration) (session *Session) {
    cluster.Acquire()
    session = &Session{
        cluster_:    cluster,
        syncTimeout: timeout,
        sockTimeout: timeout,
        poolLimit:   4096,
    }
    debugf("New session %p on cluster %p", session, cluster)
    session.SetMode(consistency, true)
    session.SetSafe(&Safe{})
    session.queryConfig.prefetch = defaultPrefetch
    return session
}

Chiasmus

mgo.Session.Copy() does the same in a much simpler way plus it preserves authentication information and other configurationof the original session and the session is set up lazily. – Hound