Error 10334 "BSONObj size is invalid" when importing a back up with mongorestore

Question

I have a MongoDB collection backup containing many small documents. The back up was produced by mongodump, but when I try to import it using mongorestore I get an error:

AssertionException handling request, closing client connection: 10334 BSONObj size: 18039019 (0x11340EB) is invalid. Size must be between 0 and 16793600(16MB)

I'm running MongoDB version 3.0.3 (from trunk).

Fireside

For future references, I took the liberty to heavily edit your question. Feel free to revert and/or edit further if you need. – Sororicide

Possible duplicate of exception: BSONObj size: -286331154 (0xEEEEEEEE) is invalid. Size must be between 0 and 16793600(16MB) – Lionhearted

I voted for closing of duplicate but my answer is posted here: https://mcmap.net/q/2029693/-exception-bsonobj-size-286331154-0xeeeeeeee-is-invalid-size-must-be-between-0-and-16793600-16mb – Lionhearted

Answer 1

Using --batchSize=100 fixes this issue for me every time.

e.g. mongorestore -d my-database --batchSize=100 ./database-dump-directory

Heliozoan

Answer 2

Basically mongoDB accept the size of the document should be less than 16MB.If you intent to use the document which is more than 16MB ,you can use gridfs. Each document takes memory power of 2 size allocation. Your application should ensure the size of bson document it is generating..Or else you can use the different data model rather than embedding all the data in one doc.

Maser

Answer 3

mongorestore send insert commands by batch in a {"applyOps", entries} document. This document is (AFAIK) limited to 16MB just like any other document.

According to the sources there are "pathological cases where the array overhead of many small operations can overflow the maximum command size". The variable oplogMaxCommandSize is used to help mongorestore to not fail on such cases. It was raised to 16.5M at some point during the 3.0... development. That was too optimistic. It was lowered back to 8M later (JIRA TOOLS-754).

If you need to, you may adjust that value yourself according to your needs. And then recompile the tools.

Sororicide