Convert mongodb return object to dictionary

Question

I'm using the bottle framework together with mongoengine. I have an orders model :

class OrderDetail(Option):
    orderDetailsQty = FloatField()

    def to_dict(self):
        return mongo_to_dict_helper(self)


class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def to_dict(self):
        orderObj = mongo_to_dict_helper(self)
        orderDetailList = []
        for orderDetail in orderObj["orderDetails"]:
            orderDetailList.append(orderDetail.__dict__)
        orderObj["OrderDetails"] = orderDetailList
        return (self)

When mongodb is queried I get an object which is then converted in to a dict by using the following function :

def mongo_to_dict_helper(obj):
    return_data = []
    for field_name in obj._fields:
        if field_name in ("id",):
            continue
        data = obj._data[field_name]

        if isinstance(obj._fields[field_name], StringField):
            return_data.append((field_name, str(data)))
        elif isinstance(obj._fields[field_name], FloatField):
            return_data.append((field_name, float(data)))
        elif isinstance(obj._fields[field_name], IntField):
            return_data.append((field_name, int(data)))
        elif isinstance(obj._fields[field_name], ListField):
            return_data.append((field_name, int(data)))
        else:
            # You can define your logic for returning elements
            pass
    return dict(return_data)

I found this function after a long search in the internet. Later found out that this function also fails while defining a member as the ListField(EmbeddedDocumentField(obj)).

I also tried writing a condition for catching the specific case of EmbeddedDocumentField :

elif isinstance(obj._fields[field_name], EmbeddedDocumentField):
    return_data.append(mongo_to_dict_helper(data))

but that didn't do any good either.

Anyone have a workaround for this issue ?

Quintus

You are trying to convert whole queryset to a dict or just one document? – Decemvir

What are you trying to achieve? – Hamby

I was trying to convert one object to json I was unaware of the to_mongo() function... – Quintus

Answer 1

What about just using to_mongo method of an object to convert it to a dict?

object.to_mongo()

Decemvir

Thats exactly what I was looking for. But still I get an error TypeError that the ObjectId is not serializable. – Quintus

Pasting a link to to_mongo() documentation will be greatly appreaceated as I cant find one anywhere. Thanks @Decemvir – Quintus

Actually, i did not find this method in the official documentation) So it is only looking into a source) – Decemvir

And about your error, try to serialize the str(ObjectId). Actually, i don't know what are you trying to do, so it is just a guess. – Decemvir

Is possible dereference all reference fields recursively using to_mongo()? – Hoagy

@Decemvir how to use this method?? from this inbox = Inbox.objects.filter(messages__to_users__in=[username]).to_mongo() i got this 'BaseQuerySet' object has no attribute 'to_mongo' – Gymnosperm

@Gymnosperm you can use a comprehension: [ob.to_mongo() for ob in query_set] – Wexler

Is there a from_mongo() as well? Does the to_mongo store metainfo to restore the object? – Isomorph

object.to_mongo() returns SON object to get a dictionary you need to call to_dict() method: object.to_mongo().to_dict() – Bestialize

Answer 2

Expanding on @alexvassel's and @z0r's answers, calling .to_mongo() converts the object to a SON instance. Once you have it, you can call its .to_dict() method to convert it to a dictionary.

For example... (qset is a queryset that's returned from mongoengine, after e.g. Posts.objects.all()).

sons = [ob.to_mongo() for ob in qset]
for son in sons:
    print str(son.to_dict())

Earache

to_mongo accepts use_db_field argument, which should be False if you have a field with a custom db_field. – Lucillalucille

as a one liner [o.to_mongo().to_dict() for o in results] which can be passed into pandas pd.DataFrame([o.to_mongo().to_dict() for o in results]) – Bazar

Answer 3

import json
json.loads(yourobj.to_json())

Michelemichelina

This is my most favorite answer in this thread, however, it returns a list of dicts and not a dict directly - even though there is just one result. In order to access one dict you have to access the lists objects like list[i]. Otherwise this is a great solution! – Kendrickkendricks

Answer 4

Extending on @alexvassel's answer, to_mongo() method returns SON object, which you can convert to dict by calling its to_dict() method

object.to_mongo().to_dict()

Bestialize

Answer 5

you can custom method to convert object to dict

class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def as_dict(self):
        return {
            "user_name": self.userName,
            "order_date": self.orderDate.strftime("%Y-%m-%d %H:%M:%S"),
        }

now you can use obj.as_dict() to dict

orders = Order.objects.all()
datas = [each.as_dict() for each in orders]

Andiron

so how about orderStatus,orderDetails and orderComments which is a EmbeddedDocumentField ,and in some case is a ReferenceField,in such case,how can it be done?? – Doe

@ken,you can define as_dict() at ReferenceField, and` "order_status":self.orderStatus.as_dict(); or, you can use aggregate,obj.objects.aggregate([{"$lookup":{"localField":"orderStatusId","foreignFiled":"_id","form":"orderStatus","as":"order_status"}}])` – Andiron

Answer 6

combining all other answers,

import json
dict = {'data':[json.loads(ob.to_json()) for ob in qset]}

Onomatology

Answer 7

There can be two scenario.

1. when query returns CommandCursor object

    **records = list(CursorObject)**

    ex - Class.objects().aggregate({...})

2. when query returns BaseQuerySet object

   **import json**

   **records = json.loads(BaseQuerySetObject.to_json())**

   ex - Class.objects().filter(..)

Bargello