mongoengine - Ignore extra fields for schema validation

Question

I am trying to query my database. Some records currently have extra fields that are not included in my model schema (by error, but I want to handle these cases). When I try to query the DB and transform the records into the schema, I get the following error:

FieldDoesNotExist
The field 'X' does not exist on the document 'Y'

Because of the extra fields in the database that differ from the schema.

Is there a way to ignore this schema validation for extra fields in mongoengine?

Personify

Answer 1

For ignoring this error when having extra fields while data loading, set strict to False in your meta dictionary.

class User(Document):
    email = StringField(required=True, unique=True)
    password = StringField()
    meta = {'strict': False}

Rattletrap

References: mongoengine-odm.readthedocs.io/… mongoengine-odm.readthedocs.io/… – Apoenzyme

Answer 2

I believe you want to use a DynamicDocument instead of a Document when defining your model and that will allow extra fields in the db schema to be ignored.

Writhe

So, what is the difference between using strict in meta as of @JFathi and DynamicDocument? – Team

@gia huy, DynamicDocument implies that it is expected behaviour in production to have extra fields. "strict" implies that it's not really okay, but that you are willing to be sloppy. For instance, while you are developing and the schema is still changing a lot, you might want to set strict to false in that development phase. – Winner

Answer 3

I think you want skip schema validation, so when you save your document

document_name.save(validate=False)

Counts

Hey, thanks for the response! This would work but I still would like the fields in my schema to be validated, I just also want to ignore any extra fields that are returned in my records. The obvious method is to use the only() on all my fields, but that isn't very nice. – Personify

Answer 4

You can extend from mon.DynamicDocument.

class AF(mon.DynamicDocument):
  meta = {
    'collection': 'af'
  }
user_id = mon.StringField(db_field='customer_user_id')

You can see from the document. A Dynamic Document class is allowing flexible, expandable and uncontrolled schemas.

Brant

Answer 5

Just define your class with DynamicDocument

class Y(DynamicDocument):
    pass

Add whatever attrs you want

o=Y()
o.attr1="abc"

Save it ;-) without error

o.save()

Eckardt