Bulk Upsert with MongoDB Java 3.0 Driver

Question

In the earlier versions of MongoDB Java drivers , to run a query and do unordered bulk upsert on the result all we had do was :

BulkWriteOperation bulk = dbCollection.initializeUnorderedBulkOperation();
    bulk.find(searchQuery).upsert().update(new BasicDBObject("$set", getDbObjectModel()));

But in version 3, with the introduction of Bson Document support and MongoCollection.bulkWrite() method how can this be done?

I tried this :

List<WriteModel<Document>> documentList = new ArrayList<>();

collection.bulkWrite(documentList, new BulkWriteOptions().ordered(false));

but, I need the upsert functionality.

Thanks.

Orthohydrogen

Answer 1

You can still use all of the functionality, it's just that BulkWrites now have a different syntax:

    MongoCollection<Document> collection = db.getCollection("sample");

    List<WriteModel<Document>> updates = Arrays.<WriteModel<Document>>asList(
        new UpdateOneModel<Document>(
                new Document(),                   // find part
                new Document("$set",1),           // update part
                new UpdateOptions().upsert(true)  // options like upsert
        )
    );

    BulkWriteResult bulkWriteResult = collection.bulkWrite(updates);

So you use the UpdateOneModel ( or for many if you want ) and set the UpdateOptions as the third argument to the constructor.

Takes some getting used to, but it's basically just building "Lists" with all the same syntax as elsewhere. I guess that's the main reason for the change.

Boxcar

Is there any official documentation for these changes in the driver? – Orthohydrogen

@AswinJoseRoy Examples? No. Unfortunately all documentation examples ( at least official ) seem to follow the older classes. This is the same story for most language drivers. For me, I found more information by searching "GitHub" for "tests" and such in the repository. But again, some "tests" use the older classes "still". So some is trial and error. Will get better, and questions like yours actually help. – Boxcar

When inserting the data at the first time, both insertedCount and ModifiedCount of bulkWriteResult are zero, is it a bug? – Bouldin

Answer 2

Here is the example using latest APIs..

for (Long entityId : entityIDs) {

    //Finder doc
    Document filterDocument = new Document();
    filterDocument.append("_id", entityId);

    //Update doc
    Document updateDocument = new Document();
    Document setDocument = new Document();
    setDocument.append("name", "xyz");
    setDocument.append("role", "abc");

    updateDocument.append("$set", setDocument);

    //Update option
    UpdateOptions updateOptions = new UpdateOptions();
    updateOptions.upsert(true); //if true, will create a new doc in case of unmatched find
    updateOptions.bypassDocumentValidation(true); //set true/false

    //Prepare list of Updates
    updateDocuments.add(
            new UpdateOneModel<Document>(
                    filterDocument,
                    updateDocument,
                    updateOptions));

}

//Bulk write options
BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
bulkWriteOptions.ordered(false);
bulkWriteOptions.bypassDocumentValidation(true);

MongoCollection<Document> mongoCollection = mongoDB.getCollection("myCollection");

BulkWriteResult bulkWriteResult = null;
try {
    //Perform bulk update
    bulkWriteResult = mongoCollection.bulkWrite(updateDocuments,
            bulkWriteOptions);
} catch (BulkWriteException e) {
    //Handle bulkwrite exception
    List<BulkWriteError> bulkWriteErrors = e.getWriteErrors();
    for (BulkWriteError bulkWriteError : bulkWriteErrors) {
        int failedIndex = bulkWriteError.getIndex();
        Long failedEntityId = entityIDs.get(failedIndex);
        System.out.println("Failed record: " + failedEntityId);
        //handle rollback
    }
}

int rowsUpdated = bulkWriteResult.getModifiedCount();

Details at: https://ashutosh-srivastav-mongodb.blogspot.in/2017/09/mongodb-bulkwrite-java-api.html

Antidromic

This works for the latest version of Java mongo driver : 3.6.0-rc0 on central repo. – Riggins

+1 it works, List<WriteModel<Document>> updateDocuments = new ArrayList<WriteModel<Document>>(); Link is broken, here is ashutosh-srivastav-mongodb.blogspot.com/2017/09/… – Rozalin

Answer 3

If you want something findAndModifyElseCreate(); That means if the document exist then update it else create it and insert the data then PLEASE FOLLOW THIS.

BasicDBObject insertInCaseDocumentNotFound = new BasicDBObject();


insertInCaseDocumentNotFound.put("field1", "value1");
insertInCaseDocumentNotFound.put("date", new Date());


MongoCollection<BasicDBObject> table = db.getCollection("collectionName",BasicDBObject.class);

BasicDBObject updateObject = new BasicDBObject();

updateObject.append("$setOnInsert", new BasicDBObject());
updateObject.append("$set", new BasicDBObject("date",new Date());

List<WriteModel<BasicDBObject>> updates = Arrays.<WriteModel<BasicDBObject>> asList(
                new UpdateOneModel<BasicDBObject>(new BasicDBObject("search_name", alert.getString("search_name")), // query for which we need to apply

                        updateObject, // update the document in case it is found
                        new UpdateOptions().upsert(true) // upsert to insert new data in case document is not found
        ));
table.bulkWrite(updates);

Cassandra