Dump Mongo Collection into JSON format

Question

Is there any way to dump mongo collection into json format? Either on the shell or using java driver.I am looking for the one with best performance.

Violate

mongoexport -d <database> -c <collection_name> --out <xxx.json> --pretty --host <host> --port <port> --username <user> --authenticationDatabase admin You can specify the host, port, username, password like this and the default authentication database is admin. – Debbidebbie

Answer 1

Mongo includes a mongoexport utility (see docs) which can dump a collection. This utility uses the native libmongoclient and is likely the fastest method.

mongoexport -d <database> -c <collection_name>

Also helpful:

-o: write the output to file, otherwise standard output is used (docs)

--jsonArray: generates a valid json document, instead of one json object per line (docs)

--pretty: outputs formatted json (docs)

Luckett

Use the -d flag to specify what database to use. – Devinne

If you want pretty printed JSON (e.g. to inspect a collection during development) use the --pretty flag: mongoexport -d mydatabase -c mycollection --pretty – Catlee

If Mongo is located on a different host, here's an example from the Mongo doc mongoexport --host mongodb1.example.net --port 37017 --username user --password "pass" --collection contacts --db marketing --out mdb1-examplenet.json – Catlett

It seems like --pretty is gone as of version 2.6.10. – Sundried

I guess this method exports in Mongodb extended JSON format. For example dates are like ISODATE("2021-12-...."). I had problems parsing this format with Apache Drill. I don't know if you can override the export to output in well-known JSON. – Solangesolano

We have loads of collections, possible to just dump all collections? – Grafton

Answer 2

Use mongoexport/mongoimport to dump/restore a collection:

Export JSON File:

mongoexport --db <database-name> --collection <collection-name> --out output.json

Import JSON File:

mongoimport --db <database-name> --collection <collection-name> --file input.json

WARNING mongoimport and mongoexport do not reliably preserve all rich BSON data types because JSON can only represent a subset of the types supported by BSON. As a result, data exported or imported with these tools may lose some measure of fidelity.

Also, http://bsonspec.org/

BSON is designed to be fast to encode and decode. For example, integers are stored as 32 (or 64) bit integers, so they don't need to be parsed to and from text. This uses more space than JSON for small integers, but is much faster to parse.

In addition to compactness, BSON adds additional data types unavailable in JSON, notably the BinData and Date data types.

Kylix

Is there an example of what “rich BSON data” would not survive a mongoexport/mongoimport round trip? – Foyer

It adds support for data types like Date and binary that aren't supported in JSON. Also, faster to encoding and decoding bsonspec.org – Kylix

Answer 3

Here's mine command for reference:

mongoexport --db AppDB --collection files --pretty --out output.json

---

On Windows 7 (MongoDB 3.4), one has to move the cmd to the place where mongod.exe and mongo.exe file resides => C:\MongoDB\Server\3.4\bin else it won't work saying it does not recongnize mongoexport command.

Telegraphese

Does anyone know of a formatter that will format normal json to the silly "single line no comma" format that MongoDB expects when importing? – Hoax

Answer 4

From the Mongo documentation:

The mongoexport utility takes a collection and exports to either JSON or CSV. You can specify a filter for the query, or a list of fields to output

Read more here: http://www.mongodb.org/display/DOCS/mongoexport

Consumption

Answer 5

Here is a little node script that I write to dump all collections in a specific database to the specified output directory...

#!/usr/bin/env node
import { MongoClient } from 'mongodb';
import { spawn } from 'child_process';
import fs from 'fs';

const DB_URI = 'mongodb://0.0.0.0:27017';
const DB_NAME = 'database-name';
const OUTPUT_DIR = 'output-directory';
const client = new MongoClient(DB_URI);

async function run() {
  try {
    await client.connect();

    const db = client.db(DB_NAME);
    const collections = await db.collections();

    if (!fs.existsSync(OUTPUT_DIR)) {
      fs.mkdirSync(OUTPUT_DIR);
    }

    collections.forEach(async (c) => {
      const name = c.collectionName;
      await spawn('mongoexport', [
        '--db',
        DB_NAME,
        '--collection',
        name,
        '--jsonArray',
        '--pretty',
        `--out=./${OUTPUT_DIR}/${name}.json`,
      ]);
    });
  } finally {
    await client.close();
    console.log(`DB Data for ${DB_NAME} has been written to ./${OUTPUT_DIR}/`);
  }
}
run().catch(console.dir);

Dollop

how does this work for non local DBs? let's say I am using ATLAS... and want to connect remotely and create a copy on my local drive? do I need to install mongodb first locally? Can it be done without? – Dyeline

@DS_web_developer just add the uri parameter https://mcmap.net/q/152224/-export-database-from-mongodb-atlas-to-the-local-machine-monogo-compass – Peoria