Mongo engine query the referencefield

Question

I have define the class statement:

from mongoengine import *
class TickDataDocument(Document):
"""
"""
    instrument_id = StringField(max_length=10, unique=True, required=True)
    tick_data = ReferenceField(TickDocument)


class TickDocument(Document):
"""
"""
    price = DecimalField(precision=2, required=True) 
    volume = LongField(required=True)  
    turnover = DecimalField(precision=2, required=True)  
    update_time = DateTimeField(unique=True, required=True)

I want to query the update_time in some period of time.

 TickDataDocument.objects(instrument_id="fa1100").filter(tick_data__ update_time__lt =datetime.datetime(2013,9,3))

but I got errors:

mongoengine.errors.InvalidQueryError: Cannot perform join in mongoDB: tick_data__update_time

How could I to five a solution to query the inner reference field in mongoengine.

Casting

can you make it two different queries ? first to get the tickdatadocuments and querying for these documents in tickDocument ? – Burress

Answer 1

As suggested in a comment, I do this in two steps (two queries):

tick_docs = list(TickDocument.object(update_time__lt)=datetime.datetime(2013,9,3))

TickDataDocument.objects(instrument_id="fa1100").filter(tick_data__in=tick_docs)

If there is any simpler solution, I'm interested as well.

Dicta

Is it a only way? If TickDocument has too many rows, maybe it occur memory issue. – Winegrower

@Winegrower yes there is, check out my answer – Precast

Answer 2

You can use the aggregate method in mongoengine and use the $lookup syntax to make your join and query, be aware the using the aggregate method returns a cursor object, not a QuerySet.

Example

TickDataDocument.objects().aggregate([
    {"$match":{"instrument_id":"fa1100"}},
    {"$lookup":{
        "from": TickDocument._get_collection_name(),
        "localField": "tick_data",
        "foreignField": "_id",
        "as": "tick"            
    }},
    {"$match":{
        "tick.update_time":{"$lt":datetime.datetime(2013,9,3)}
    }}
])

Precast