PyMongo/Mongoengine equivalent of mongodump

Question

Is there an equivalent function in PyMongo or mongoengine to MongoDB's mongodump? I can't seem to find anything in the docs.

Use case: I need to periodically backup a remote mongo database. The local machine is a production server that does not have mongo installed, and I do not have admin rights, so I can't use subprocess to call mongodump. I could install the mongo client locally on a virtualenv, but I'd prefer an API call.

Thanks a lot :-).

Blitzkrieg

Pymongo and therefore MongoEngine only connect to the mongod process. However, a way around your problem might be to simply run mongodump on a remote machine, because you can connect to any remote database with mongodump/restore etc. You don't need to run mondodump on the machine that is hosting the db. Make sense? – Luzon

Meh. I guess it's easier to just extract all documents and then build a BSON file. – Blitzkrieg

Answer 1

For my relatively small small database, I eventually used the following solution. It's not really suitable for big or complex databases, but it suffices for my case. It dumps all documents as a json to the backup directory. It's clunky, but it does not rely on other stuff than pymongo.

from os.path import join
import pymongo
from bson.json_utils import dumps

def backup_db(backup_db_dir):
    client = pymongo.MongoClient(host=<host>, port=<port>)
    database = client[<db_name>]
    authenticated = database.authenticate(<uname>,<pwd>)
    assert authenticated, "Could not authenticate to database!"
    collections = database.collection_names()
    for i, collection_name in enumerate(collections):
        col = getattr(database,collections[i])
        collection = col.find()
        jsonpath = collection_name + ".json"
        jsonpath = join(backup_db_dir, jsonpath)
        with open(jsonpath, 'wb') as jsonfile:
            jsonfile.write(dumps(collection))

Blitzkrieg

thanks for posting this! Does this differ in any way from cli mongodump? Will the resulting json file be usable with mongorestore ? – Christean

You could consider iterating over the result of col.find() if you don't intend to hold all the documents in memory at once. You did mention "relatively small small database". – Ruwenzori

can we store as bson? – Cecilla

This should be from bson.json_util import dumps instead of utils.!!!!!! – Cecilla

I found this solution helpful, but i also want backup in single file. Any suggestions how can i archive in single file as export and import that file to restore database? – Disadvantage

This is good, but will not create .bson e .metadata.json files. So I think restore using mongorestore will not be able. – Simms

Answer 2

The accepted answer is not working anymore. Here is a revised code:

from os.path import join
import pymongo
from bson.json_util import dumps

def backup_db(backup_db_dir):
    client = pymongo.MongoClient(host=..., port=..., username=..., password=...)
    database = client[<db_name>]
    collections = database.collection_names()

    for i, collection_name in enumerate(collections):
        col = getattr(database,collections[i])
        collection = col.find()
        jsonpath = collection_name + ".json"
        jsonpath = join(backup_db_dir, jsonpath)
        with open(jsonpath, 'wb') as jsonfile:
            jsonfile.write(dumps(collection).encode())


backup_db('.')

Purse

Why do you use .encode here? – Lipase

don't remember why I did that, probably because it is written as bytes. Try to remove the encode and see if it works :) – Purse

I believe you don't need it, that's why I was curious. Thank you! – Lipase

The collection_names() is deprecated in the version 3.7.0. Instead use database.list_collection_names() – Urdu