Mongo DB 4.0 Transactions With Mongoose & NodeJs, Express

Question

I am developing an application where I am using MongoDB as database with Nodejs + Express in application layer, I have two collections, namely

1. users
2. transactions

Here i have to update wallet of thousands of users with some amount and if successful create a new document with related info for each transaction, This is My code :

 userModel.update({_id : ObjectId(userId)}, {$inc : {wallet : 500}}, function (err, creditInfo) {
    if(err){
        console.log(err);                            
    }
    if(creditInfo.nModified > 0) {
        newTransModel = new transModel({
            usersId: ObjectId(userId),            
            amount: winAmt,         
            type: 'credit',           
        }); 
        newTransModel.save(function (err, doc) {
            if(err){
                Cb(err); 
            }
        });
    }                            
});

but this solution is not atomic there is always a possibility of user wallet updated with amount but related transaction not created in transactions collection resulting in financial loss.

I have heard that recently MongoDB has added Transactions support in its 4.0 version, I have read the MongoDB docs but couldn't get it to successfully implement it with mongoose in Node.js, can anyone tell me how this above code be reimplemented using the latest Transactions feature of MongoDB which have these functions

Session.startTransaction()
Session.abortTransaction()
Session.commitTransaction()

MongoDB Docs : Click Here

Arango

Answer 1

with mongoose in Node.js, can anyone tell me how this above code be reimplemented using the latest Transactions feature

To use MongoDB multi-documents transactions support in mongoose you need version greater than v5.2. For example:

npm install [email protected]

Mongoose transactional methods returns a promise rather than a session which would require to use await. See:

• Transactions in Mongoose
• Blog: A Node.JS Perspective on MongoDB 4.0: Transactions

For example, altering the example on the resource above and your example, you can try:

const User = mongoose.model('Users', new mongoose.Schema({
  userId: String, wallet: Number
}));
const Transaction = mongoose.model('Transactions', new mongoose.Schema({
  userId: ObjectId, amount: Number, type: String
}));

await updateWallet(userId, 500);

async function updateWallet(userId, amount) {
  const session = await User.startSession();
  session.startTransaction();
  try {
    const opts = { session };
    const A = await User.findOneAndUpdate(
                    { _id: userId }, { $inc: { wallet: amount } }, opts);

    const B = await Transaction(
                    { usersId: userId, amount: amount, type: "credit" })
                    .save(opts);

    await session.commitTransaction();
    session.endSession();
    return true;
  } catch (error) {
    // If an error occurred, abort the whole transaction and
    // undo any changes that might have happened
    await session.abortTransaction();
    session.endSession();
    throw error; 
  }
}

is not atomic there is always a possibility of user wallet updated with amount but related transaction not created in transactions collection resulting in financial loss

You should also consider changing your MongoDB data models. Especially if the two collections are naturally linked. See also Model data for Atomic Operations for more information.

An example model that you could try is Event Sourcing model. Create a transaction entry first as an event, then recalculate the user's wallet balance using aggregation.

For example:

{tranId: 1001, fromUser:800, toUser:99, amount:300, time: Date(..)}
{tranId: 1002, fromUser:77, toUser:99, amount:100, time: Date(..)}

Then introduce a process to calculate the amount for each users per period as a cache depending on requirements (i.e. per 6 hours). You can display the current user's wallet balance by adding:

• The last cached amount for the user
• Any transactions for the user occur since the last cached amount. i.e. 0-6 hours ago.

Leonilaleonine

there are two things about this 1) await updateWallet(userId, 500); <<< this gives an error saying await needs to be in a async function , top level await is not allowed 2) as i am doing this in loop async eachseries i.e updating many users wallet , i am getting writeconflit error of transactions & when i call next iteration, also using a finally block after try catch also doesn't work – Arango

This is an example, you'd have to adjust for your own use case. 1) Make sure you create an async function as shown above. 2) You can try performing bulkWrite() within a transaction instead. – Leonilaleonine

thanks for assistance, i already implemented this successfully, the second error was my coding mistake not an issue of the code. – Arango

just you edit your answer and edit this >> User.db.startSession(); to Mymodel.startSession() ........it says db is undefined, we can use any mongoose model to start session its not scoped to that model once created. – Arango

Ah yes, I was trying to give emphasis that the session for for the database not on the collection, but made a mistake. Fixed now. thanks. – Leonilaleonine

question: I don't see any use of .endSession(), could somebody care to explain whether we should need it or not? – Needlefish

@WanBachtiar how oo i add new: true as well as sessions in 3rd param? – Rodrickrodrigez

@Needlefish ..... like in mysql even this works on a session which mongodb creates, eveithing is registered on a session and only written to db when you call committ, its a good practice to end the session when your work is done like we do end connection in mysql, its good for your server resources too & security. – Arango

Well, in that case, @WanBachtiar, it would a good idea to add a finally block to endSession(). It would a) save a code line b) would guarantee the execution. – Needlefish

@GauravKumar, that comment was supposed to go to Kannan T right ? – Leonilaleonine

@Yogesh, that would be your choice on how to structure your code. The example above is to show how a transaction based on a session is started and committed/aborted. – Leonilaleonine

@KannanT ..... i think you can just declare an object in their third parameter something like this { session : session , new : true } – Arango

@WanBachtiar ...yes it was for kannan T......mentioned you by mistake !! – Arango

MongooseError: Callback must be a function, got [object Object] I have got this error. What can I do for this – Durrett

@WanBachtiar Can we handle concurrent error with transaction ? I mean if I update 2 times to 1 record so fast, and it's updated to the same __v of record so the concurrency updating is happen, if it's throw to catch, can I abort transaction or the concurrent error can happen even transaction is not committed ? – Ere

Would this still work properly if the two queries would be executed in parallel using await Promise.all? – Drubbing

Fantastic, thanks ! – Artiodactyl

I have just one question, don't we have to start the session of Transaction model too? As we have done for User model. – Madonnamadora