Convert Django Model object to dict with all of the fields intact

Question

How does one convert a django Model object to a dict with all of its fields? All ideally includes foreign keys and fields with editable=False.

Let me elaborate. Let's say I have a django model like the following:

from django.db import models

class OtherModel(models.Model): pass

class SomeModel(models.Model):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

In the terminal, I have done the following:

other_model = OtherModel()
other_model.save()
instance = SomeModel()
instance.normal_value = 1
instance.readonly_value = 2
instance.foreign_key = other_model
instance.save()
instance.many_to_many.add(other_model)
instance.save()

I want to convert this to the following dictionary:

{'auto_now_add': datetime.datetime(2015, 3, 16, 21, 34, 14, 926738, tzinfo=<UTC>),
 'foreign_key': 1,
 'id': 1,
 'many_to_many': [1],
 'normal_value': 1,
 'readonly_value': 2}

---

Questions with unsatisfactory answers:

Django: Converting an entire set of a Model's objects into a single dictionary

How can I turn Django Model objects into a dictionary and still have their foreign keys?

Rollick

you can declare a method called to_dict and handle it the way you want. – Housebound

@Housebound yes, I could. The question is how to create such a method. Manually constructing a dictionary from all of the fields of the model is not a suitable answer. – Rollick

I'd leverage an existing ReST library like Django Rest Framework, Tastypie or Piston since they all provide mechanisms to convert django model instances into primitives for serialization. If you're more curious how, you can look through their code, but it's mostly walking through the model's _meta definitions to find the fields associated with the model and retrieve their values on the instance. – Quackery

Answer 1

There are many ways to convert an instance to a dictionary, with varying degrees of corner case handling and closeness to the desired result.

---

1. instance.__dict__

instance.__dict__

which returns

{'_foreign_key_cache': <OtherModel: OtherModel object>,
 '_state': <django.db.models.base.ModelState at 0x7ff0993f6908>,
 'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is by far the simplest, but is missing many_to_many, foreign_key is misnamed, and it has two unwanted extra things in it.

---

2. model_to_dict

from django.forms.models import model_to_dict
model_to_dict(instance)

which returns

{'foreign_key': 2,
 'id': 1,
 'many_to_many': [<OtherModel: OtherModel object>],
 'normal_value': 1}

This is the only one with many_to_many, but is missing the uneditable fields.

---

3. model_to_dict(..., fields=...)

from django.forms.models import model_to_dict
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])

which returns

{'foreign_key': 2, 'id': 1, 'normal_value': 1}

This is strictly worse than the standard model_to_dict invocation.

---

4. query_set.values()

SomeModel.objects.filter(id=instance.id).values()[0]

which returns

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is the same output as instance.__dict__ but without the extra fields. foreign_key_id is still wrong and many_to_many is still missing.

---

5. Custom Function

The code for django's model_to_dict had most of the answer. It explicitly removed non-editable fields, so removing that check and getting the ids of foreign keys for many to many fields results in the following code which behaves as desired:

from itertools import chain

def to_dict(instance):
    opts = instance._meta
    data = {}
    for f in chain(opts.concrete_fields, opts.private_fields):
        data[f.name] = f.value_from_object(instance)
    for f in opts.many_to_many:
        data[f.name] = [i.id for i in f.value_from_object(instance)]
    return data

While this is the most complicated option, calling to_dict(instance) gives us exactly the desired result:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

---

6. Use Serializers

Django Rest Framework's ModelSerializer allows you to build a serializer automatically from a model.

from rest_framework import serializers
class SomeModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = SomeModel
        fields = "__all__"

SomeModelSerializer(instance).data

returns

{'auto_now_add': '2018-12-20T21:34:29.494827Z',
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

This is almost as good as the custom function, but auto_now_add is a string instead of a datetime object.

---

Bonus Round: better model printing

If you want a django model that has a better python command-line display, have your models child-class the following:

from django.db import models
from itertools import chain

class PrintableModel(models.Model):
    def __repr__(self):
        return str(self.to_dict())

    def to_dict(instance):
        opts = instance._meta
        data = {}
        for f in chain(opts.concrete_fields, opts.private_fields):
            data[f.name] = f.value_from_object(instance)
        for f in opts.many_to_many:
            data[f.name] = [i.id for i in f.value_from_object(instance)]
        return data

    class Meta:
        abstract = True

So, for example, if we define our models as such:

class OtherModel(PrintableModel): pass

class SomeModel(PrintableModel):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

Calling SomeModel.objects.first() now gives output like this:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

Rollick

Thanks for this answer! You might change the isinstance test in solution #5 (and the bonus) to if f.many_to_many. – Lane

@Lane I modeled that code off of Django's model_to_dict code, which uses isinstance. I'm not sure why they made this choice but there may be a good reason for it (such as the many_to_many property being introduced in a later version) – Rollick

would it also return @property fields ? – Leis

I wonder how would these methods treat the annotated/aggregated fields? – Bartolommeo

Something I do is check for get_FOO_display and return that value instead of whatever value may actually be there. – Jehoshaphat

Briiliant answer! Thanks! There seems to be a typo, however, in the custom function, to_dict: for f in opts.many_to_many: data[f.name] = [i.id for i in f.value_from_object(u)] The u should be an i. – Mattern

@Mattern Fixed. (The u needed to be instance) – Rollick

I am sorry, but if your field is of GenericForeignKey, the Custom Function will fail (no value_from_object on GenericForeignKey field type). So you have to check if f is not of instance GenericForeignKey. – Craggy

@Craggy feel free to make an edit to support genericforeignkeys – Rollick

An important thing about model_to_dict: It does not just present data already in the object. It might make a database query. If your object is a partial one fetched with QuerySet.raw() it will go get all the other columns you deliberately left out. – Derryberry

anyone going to use the drf serializers for converting thousand of millions of rows in a loop or something, it might not be a good idea. serializers are very slow. hakibenita.com/django-rest-framework-slow – Virgievirgil

An important thing about model_to_dict: It does not present data for model field that has attribute: editable = False. The best solution is to use custom method to_dict. Thank a lot. – Carilla

I was able to just call values = MyModel.objects.all().values().first(). – Woodley

Answer 2

I found a neat solution to get to result:

Suppose you have an model object o:

Just call:

type(o).objects.filter(pk=o.pk).values().first()

Flourish

This is just option #4 in my answer – Rollick

Answer 3

Simplest way,

1. If your query is Model.Objects.get():

   get() will return single instance so you can direct use __dict__ from your instance.

   model_dict = Model.Objects.get().__dict__

2. for filter()/all():

   all()/filter() will return list of instances so you can use values() to get list of objects.

   model_values = Model.Objects.all().values()

Giustino

model_dict = Model.Objects.get().__dict__ Gives an item with a key called _state along with the rest of the items – Flare

Answer 4

just vars(obj) , it will state the whole values of the object

>>> obj_attrs = vars(obj)
>>> obj_attrs
 {'_file_data_cache': <FileData: Data>,
  '_state': <django.db.models.base.ModelState at 0x7f5c6733bad0>,
  'aggregator_id': 24,
  'amount': 5.0,
  'biller_id': 23,
  'datetime': datetime.datetime(2018, 1, 31, 18, 43, 58, 933277, tzinfo=<UTC>),
  'file_data_id': 797719,
 }

You can add this also

>>> keys = obj_attrs.keys()
>>> temp = [obj_attrs.pop(key) if key.startswith('_') else None for key in keys]
>>> del temp
>>> obj_attrs
   {
    'aggregator_id': 24,
    'amount': 5.0,
    'biller_id': 23,
    'datetime': datetime.datetime(2018, 1, 31, 18, 43, 58, 933277, tzinfo=<UTC>),
    'file_data_id': 797719,
   }

Triolein

Answer 5

Update

The newer aggregated answer posted by @zags is more complete and elegant than my own. Please refer to that answer instead.

Original

If you are willing to define your own to_dict method like @karthiker suggested, then that just boils this problem down to a sets problem.

>>># Returns a set of all keys excluding editable = False keys
>>>dict = model_to_dict(instance)
>>>dict

{u'id': 1L, 'reference1': 1L, 'reference2': [1L], 'value': 1}


>>># Returns a set of editable = False keys, misnamed foreign keys, and normal keys
>>>otherDict = SomeModel.objects.filter(id=instance.id).values()[0]
>>>otherDict

{'created': datetime.datetime(2014, 2, 21, 4, 38, 51, tzinfo=<UTC>),
 u'id': 1L,
 'reference1_id': 1L,
 'value': 1L,
 'value2': 2L}

We need to remove the mislabeled foreign keys from otherDict.

To do this, we can use a loop that makes a new dictionary that has every item except those with underscores in them. Or, to save time, we can just add those to the original dict since dictionaries are just sets under the hood.

>>>for item in otherDict.items():
...    if "_" not in item[0]:
...            dict.update({item[0]:item[1]})
...
>>>

Thus we are left with the following dict:

>>>dict
{'created': datetime.datetime(2014, 2, 21, 4, 38, 51, tzinfo=<UTC>),
 u'id': 1L,
 'reference1': 1L,
 'reference2': [1L],
 'value': 1,
 'value2': 2L}

And you just return that.

On the downside, you can't use underscores in your editable=false field names. On the upside, this will work for any set of fields where the user-made fields do not contain underscores.

This is not the best way of doing this, but it could work as a temporary solution until a more direct method is found.

For the example below, dict would be formed based on model_to_dict and otherDict would be formed by filter's values method. I would have done this with the models themselves, but I can't get my machine to accept otherModel.

>>> import datetime
>>> dict = {u'id': 1, 'reference1': 1, 'reference2': [1], 'value': 1}
>>> otherDict = {'created': datetime.datetime(2014, 2, 21, 4, 38, 51), u'id': 1, 'reference1_id': 1, 'value': 1, 'value2': 2}
>>> for item in otherDict.items():
...     if "_" not in item[0]:
...             dict.update({item[0]:item[1]})
...
>>> dict
{'reference1': 1, 'created': datetime.datetime(2014, 2, 21, 4, 38, 51), 'value2': 2, 'value': 1, 'id': 1, 'reference2': [1]}
>>>

That should put you in a rough ballpark of the answer to your question, I hope.

Toniatonic

Not sure what you're trying to use re for here. If it is to filter out keys with underscores in them, this is neither correct code nor correct behavior. re.match("_", "reference1_id") returns None and legitimate columns in the database may have underscores in their names. – Rollick

re.match("_", "reference1_id") does return None, it should have been: re.match(".*_.*", "reference1_id") – Toniatonic

I made some changes to remove the bad example and include a better one. I also changed some things to express that this would be a temporary solution for a subset of all models. I have no idea what you would do for models with underscores in their editable=false fields. I was just trying to provide something you might be able to work with until a more canon solution could be delivered. – Toniatonic

Maybe use "_" in string rather than re in that case. – Rollick

Yes, that would be a much easier way of doing it. It had not occurred to me to use it in this way, but it makes perfectly good sense now. I've changed the answer to use in instead of re. – Toniatonic

Answer 6

@Zags solution was gorgeous!

I would add, though, a condition for datefields in order to make it JSON friendly.

Bonus Round

If you want a django model that has a better python command-line display, have your models child class the following:

from django.db import models
from django.db.models.fields.related import ManyToManyField

class PrintableModel(models.Model):
    def __repr__(self):
        return str(self.to_dict())

    def to_dict(self):
        opts = self._meta
        data = {}
        for f in opts.concrete_fields + opts.many_to_many:
            if isinstance(f, ManyToManyField):
                if self.pk is None:
                    data[f.name] = []
                else:
                    data[f.name] = list(f.value_from_object(self).values_list('pk', flat=True))
            elif isinstance(f, DateTimeField):
                if f.value_from_object(self) is not None:
                    data[f.name] = f.value_from_object(self).timestamp()
            else:
                data[f.name] = None
            else:
                data[f.name] = f.value_from_object(self)
        return data

    class Meta:
        abstract = True

So, for example, if we define our models as such:

class OtherModel(PrintableModel): pass

class SomeModel(PrintableModel):
    value = models.IntegerField()
    value2 = models.IntegerField(editable=False)
    created = models.DateTimeField(auto_now_add=True)
    reference1 = models.ForeignKey(OtherModel, related_name="ref1")
    reference2 = models.ManyToManyField(OtherModel, related_name="ref2")

Calling SomeModel.objects.first() now gives output like this:

{'created': 1426552454.926738,
'value': 1, 'value2': 2, 'reference1': 1, u'id': 1, 'reference2': [1]}

Reprint

If you want to convert to and from JSON, you should look into Django Rest Framework or use something like this: https://mcmap.net/q/41626/-how-can-i-overcome-quot-datetime-datetime-not-json-serializable-quot – Rollick

Sure! But this small change to your code add a great deal of convenience! – Reprint

Answer 7

The easier way is to just use pprint, which is in base Python

import pprint
item = MyDjangoModel.objects.get(name = 'foo')
pprint.pprint(item.__dict__, indent = 4)

This gives output that looks similar to json.dumps(..., indent = 4) but it correctly handles the weird data types that might be embedded in your model instance, such as ModelState and UUID, etc.

Tested on Python 3.7

Libbie

Answer 8

I faced this problem when I tried to convert a django site to an API using django-rest framework. Normally django returns three types of objects from the database. They include a queryset, a model instance and a paginator object. In my case these were the ones that needed converting.

Queryset

A queryset is like a list of of model objects in django. Here is the code to convert it into a dict.

model_data=Model.object.all()# This returns a queryset object
model_to_dict=[model for model in model_data.values()]
return Response(model_to_dict,status=status.HTTP_200_OK)

Model Instance

A model instance is a single object of a model.

model_instance=Model.objects.get(pk=1)# This will return only a single model object
model_to_dict=model_to_dict(model_instance)
return Response(model_to_dict,status=status.HTTP_200_OK)

Paginator object

A paginator object is an object that contains model objects of a particular page.

model_queryset=Model.objects.all()
paginator = Paginator(model_queryset, 10)
try:
    selected_results = paginator.page(page)
except Exception:
    selected_results=result
paginator_to_dict=list(selected_results.object_list.values())
return Response(selected_results,status=status.HTTP_200_OK)

At least that is how I solved it.

Samuel

Answer 9

(did not mean to make the comment)

Ok, it doesn't really depend on types in that way. I may have mis-understood the original question here so forgive me if that is the case. If you create serliazers.py then in there you create classes that have meta classes.

Class MyModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = modelName
        fields =('csv','of','fields')

Then when you get the data in the view class you can:

model_data - Model.objects.filter(...)
serializer = MyModelSerializer(model_data, many=True)
return Response({'data': serilaizer.data}, status=status.HTTP_200_OK)

That is pretty much what I have in a vareity of places and it returns nice JSON via the JSONRenderer.

As I said this is courtesy of the DjangoRestFramework so it's worth looking into that.

Venereal

Answer 10

Lots of interesting solutions here. My solution was to add an as_dict method to my model with a dict comprehension.

def as_dict(self):
    return dict((f.name, getattr(self, f.name)) for f in self._meta.fields)

As a bonus, this solution paired with an list comprehension over a query makes for a nice solution if you want export your models to another library. For example, dumping your models into a pandas dataframe:

pandas_awesomeness = pd.DataFrame([m.as_dict() for m in SomeModel.objects.all()])

Substage

This works fine for value fields like strings and ints, but will have some problems with foreign keys and even more with many to many fields – Rollick

Very good point! Especially for many to many. One would want to put some conditionals in to handle those cases appropriately, or limit this solution to simple models. Thanks. – Substage

Answer 11

I like to convert model instances to dict for snapshot testing, here is how I do it:

Note: there is the camelize option because if the api response returns the objects carmelized, it's better to keep all snapshots consistent, either from model instances or api calls.

from rest_framework import serializers
from djangorestframework_camel_case.util import camelize as _camelize

def model_to_dict(instance, camelize=False):
    """
    Convert a model instance to dict.
    """
    class Serializer(serializers.ModelSerializer):
        class Meta:
            model = type(instance)
            fields = "__all__"
    data = Serializer(instance).data
    if camelize:
        data = _camelize(data)
    # convert from ordered dict to dict
    return dict(data)

Tanana

Answer 12

Too many great answers already, but none of them mention Django's built-in serializers.

Django Rest Framework's ModelSerializer is mentioned several times, but that's an extra dependency.

The built-in python.Serializer looks like a good candidate. From the docstring:

A Python "serializer". Doesn't do much serializing per se -- just converts to and from basic Python data types (lists, dicts, strings, etc.). Useful as a basis for other serializers.

Here's how it would work:

from django.core.serializers.python import Serializer

instance = ...  # create instance as in OP

serialized_instance = Serializer().serialize([instance])[0]

The output is slightly different from the requested dict, but the desired fields are all there. For example:

{
    'fields': {
        'auto_now_add': datetime.datetime(2023, 3, 21, 12, 12, 27, 270255, tzinfo=datetime.timezone.utc),
        'foreign_key': 1,
        'many_to_many': [1],
        'normal_value': 1,
        'readonly_value': 2,
    },
    'model': 'various.somemodel',
    'pk': 1,
}

This can be restructured to match the dict from the OP, if necessary. For example:

instance_dict = {
    'id': serialized_instance['pk'],
    **serialized_instance['fields'],
}

This approach also works well if you've got a queryset that has already been evaluated into a list of objects.

Erective

Answer 13

Maybe this help you. May this not covert many to many relantionship, but es pretty handy when you want to send your model in json format.

def serial_model(modelobj):
  opts = modelobj._meta.fields
  modeldict = model_to_dict(modelobj)
  for m in opts:
    if m.is_relation:
        foreignkey = getattr(modelobj, m.name)
        if foreignkey:
            try:
                modeldict[m.name] = serial_model(foreignkey)
            except:
                pass
  return modeldict

Bughouse

Answer 14

Best solution you have ever see.

Convert django.db.models.Model instance and all related ForeignKey, ManyToManyField and @Property function fields into dict.

"""
Convert django.db.models.Model instance and all related ForeignKey, ManyToManyField and @property function fields into dict.
Usage:
    class MyDjangoModel(... PrintableModel):
        to_dict_fields = (...)
        to_dict_exclude = (...)
        ...
    a_dict = [inst.to_dict(fields=..., exclude=...) for inst in MyDjangoModel.objects.all()]
"""
import typing

import django.core.exceptions
import django.db.models
import django.forms.models


def get_decorators_dir(cls, exclude: typing.Optional[set]=None) -> set:
    """
    Ref: https://mcmap.net/q/48804/-how-can-i-introspect-properties-and-model-fields-in-django
    :param exclude: set or None
    :param cls:
    :return: a set of decorators
    """
    default_exclude = {"pk", "objects"}
    if not exclude:
        exclude = default_exclude
    else:
        exclude = exclude.union(default_exclude)

    return set([name for name in dir(cls) if name not in exclude and isinstance(getattr(cls, name), property)])


class PrintableModel(django.db.models.Model):

    class Meta:
        abstract = True

    def __repr__(self):
        return str(self.to_dict())

    def to_dict(self, fields: typing.Optional[typing.Iterable]=None, exclude: typing.Optional[typing.Iterable]=None):
        opts = self._meta
        data = {}

        # support fields filters and excludes
        if not fields:
            fields = set()
        else:
            fields = set(fields)
        default_fields = getattr(self, "to_dict_fields", set())
        fields = fields.union(default_fields)

        if not exclude:
            exclude = set()
        else:
            exclude = set(exclude)
        default_exclude = getattr(self, "to_dict_exclude", set())
        exclude = exclude.union(default_exclude)

        # support syntax "field__childField__..."
        self_fields = set()
        child_fields = dict()
        if fields:
            for i in fields:
                splits = i.split("__")
                if len(splits) == 1:
                    self_fields.add(splits[0])
                else:
                    self_fields.add(splits[0])

                    field_name = splits[0]
                    child_fields.setdefault(field_name, set())
                    child_fields[field_name].add("__".join(splits[1:]))

        self_exclude = set()
        child_exclude = dict()
        if exclude:
            for i in exclude:
                splits = i.split("__")
                if len(splits) == 1:
                    self_exclude.add(splits[0])
                else:
                    field_name = splits[0]
                    if field_name not in child_exclude:
                        child_exclude[field_name] = set()
                    child_exclude[field_name].add("__".join(splits[1:]))

        for f in opts.concrete_fields + opts.many_to_many:
            if self_fields and f.name not in self_fields:
                continue
            if self_exclude and f.name in self_exclude:
                continue

            if isinstance(f, django.db.models.ManyToManyField):
                if self.pk is None:
                    data[f.name] = []
                else:
                    result = []
                    m2m_inst = f.value_from_object(self)
                    for obj in m2m_inst:
                        if isinstance(PrintableModel, obj) and hasattr(obj, "to_dict"):
                            d = obj.to_dict(
                                fields=child_fields.get(f.name),
                                exclude=child_exclude.get(f.name),
                            )
                        else:
                            d = django.forms.models.model_to_dict(
                                obj,
                                fields=child_fields.get(f.name),
                                exclude=child_exclude.get(f.name)
                            )
                        result.append(d)
                    data[f.name] = result
            elif isinstance(f, django.db.models.ForeignKey):
                if self.pk is None:
                    data[f.name] = []
                else:
                    data[f.name] = None
                    try:
                        foreign_inst = getattr(self, f.name)
                    except django.core.exceptions.ObjectDoesNotExist:
                        pass
                    else:
                        if isinstance(foreign_inst, PrintableModel) and hasattr(foreign_inst, "to_dict"):
                            data[f.name] = foreign_inst.to_dict(
                                fields=child_fields.get(f.name),
                                exclude=child_exclude.get(f.name)
                            )
                        elif foreign_inst is not None:
                            data[f.name] = django.forms.models.model_to_dict(
                                foreign_inst,
                                fields=child_fields.get(f.name),
                                exclude=child_exclude.get(f.name),
                            )

            elif isinstance(f, (django.db.models.DateTimeField, django.db.models.DateField)):
                v = f.value_from_object(self)
                if v is not None:
                    data[f.name] = v.isoformat()
                else:
                    data[f.name] = None
            else:
                data[f.name] = f.value_from_object(self)

        # support @property decorator functions
        decorator_names = get_decorators_dir(self.__class__)
        for name in decorator_names:
            if self_fields and name not in self_fields:
                continue
            if self_exclude and name in self_exclude:
                continue

            value = getattr(self, name)
            if isinstance(value, PrintableModel) and hasattr(value, "to_dict"):
                data[name] = value.to_dict(
                    fields=child_fields.get(name),
                    exclude=child_exclude.get(name)
                )
            elif hasattr(value, "_meta"):
                # make sure it is a instance of django.db.models.fields.Field
                data[name] = django.forms.models.model_to_dict(
                    value,
                    fields=child_fields.get(name),
                    exclude=child_exclude.get(name),
                )
            elif isinstance(value, (set, )):
                data[name] = list(value)
            else:
                data[name] = value

        return data

https://gist.github.com/shuge/f543dc2094a3183f69488df2bfb51a52

Charlesettacharleston

Answer 15

The answer from @zags is comprehensive and should suffice but the #5 method (which is the best one IMO) throws an error so I improved the helper function.

As the OP requested for converting many_to_many fields into a list of primary keys rather than a list of objects, I enhanced the function so the return value is now JSON serializable - by converting datetime objects into str and many_to_many objects to a list of id's.

import datetime

def ModelToDict(instance):
    '''
    Returns a dictionary object containing complete field-value pairs of the given instance

    Convertion rules:

        datetime.date --> str
        many_to_many --> list of id's

    '''

    concrete_fields = instance._meta.concrete_fields
    m2m_fields = instance._meta.many_to_many
    data = {}

    for field in concrete_fields:
        key = field.name
        value = field.value_from_object(instance)
        if type(value) == datetime.datetime:
            value = str(field.value_from_object(instance))
        data[key] = value

    for field in m2m_fields:
        key = field.name
        value = field.value_from_object(instance)
        data[key] = [rel.id for rel in value]

    return data

Burcham

What's the error you get? I'm happy to update the answer – Rollick

Currently functionality of the loops through concrete_fields and m2m_fields look identical, so assuming the m2m_fields loop has an incorrect implementation here. – Wegner

@Rollick the error is AttributeError: 'list' object has no attribute 'values_list' which I could not find the reason behind it. Im using Django 2.1.1 – Burcham

@daniel-himmelstein thank you for pointing out, I fixed the code. the reason for identical loops was due to performing different operations in my local code and I forgot to optimize it for the SO answer. – Burcham

@ArminHemati Django changed the implementation of field.value_from_object and as a result, of model_to_dict. I've updated section 5 of my answer to reflect this. – Rollick

Answer 16

To convert a model to a dictionary and retain all ForiegnKey model relationships. I used the following:

Without Verbose Name

from django.forms.models import model_to_dict

instance = MyModel.objects.get(pk=1) # EXAMPLE

instance_dict = {key: getattr(instance, key) for key in model_to_dict(instance).keys()}

Output

{'foreign_key': [<OtherModel: OtherModel object>],
 'id': 1,
 'many_to_many': [<OtherModel: OtherModel object>],
 'normal_value': 1}

This can be useful if you want to display the __str__() value in your template for foreign key relationships.

Including keyword arguments fields= and exclude= into model_to_dict(instance, [...]) enables you to filter specific fields.

With Verbose Name

from django.forms.models import model_to_dict

instance = MyModel.objects.get(pk=1) # EXAMPLE

instance_dict = {instance._meta.get_field(key).verbose_name if hasattr(instance._meta.get_field(key), 'verbose_name') else key: getattr(instance, key) for key in model_to_dict(instance).keys()}

Example Output (If there were verbose names for given example)

{'Other Model:': [<OtherModel: OtherModel object>],
 'id': 1,
 'My Other Model:': [<OtherModel: OtherModel object>],
 'Normal Value:': 1}

Evita

Answer 17

I have used next function to convert model to dict

def model_to_dict(obj):
    return {x: obj.__dict__[x] for x in obj.__dict__ if x in {y.column for y in obj._meta.fields}}

examples

{'id': 8985,
 'title': 'Dmitro',
 'email_address': 'it9+8985@localhost',
 'workspace_id': 'it9',
 'archived': False,
 'deleted': False,
 'inbox': False,
 'read': True,
 'created_at': datetime.datetime(2022, 5, 5, 16, 55, 29, 791844, tzinfo=    <UTC>),
 'creator': 'An So',
 'last_message_id': 500566,
 'stat_data': {'count_messages': 1, 'count_attachments': 0},
 'stat_dirty': False,
 'assign_to_id': None,
 'assigned_at': None,
 'assignment_note': None,
 'initial_last_update_ts': 1651769728,
 'renamed_manually': False,
 'unread_timestamp': datetime.datetime(2022, 5, 5, 16, 55, 29, 842507, tzinfo=<UTC>)}

{'id': 6670,
 'email_id': 473962,
 'message_id': 500620,
 'filename': 'Screenshot.png',
 'size': 6076854,
 'mimetype': 'image/png',
 'aws_key': 'dev/RLpdcza46KFpITDWO_kv_fg2732waccB43z5RmT9/Screenshot.png',
 'aws_key1': '',
 'aws_key_thumb': 'dev/iaCdvcZmUKq-gJim7HT33ID46Ng4WOdxx-TdVuIU/f4b0db49-7f2d-4def-bdc1-8e394f98727f.png',
 's3stored_file_id': 4147}

Jared

Answer 18

I created a little snippet that utilizes django's model_to_dict but traverses through the relationships of an object. For circular dependencies it terminates the recursion and puts a string referencing the dependency object. You could probably extend this to include non_editable fields.

I use it to create model snapshots during testing.

from itertools import chain

from django.db.models.fields.files import FileField, ImageField
from django.forms.models import model_to_dict


def get_instance_dict(instance, already_passed=frozenset()):
    """Creates a nested dict version of a django model instance
    Follows relationships recursively, circular relationships are terminated by putting
    a model identificator `{model_name}:{instance.id}`.
    Ignores image and file fields."""
    instance_dict = model_to_dict(
        instance,
        fields=[
            f
            for f in instance._meta.concrete_fields
            if not isinstance(f, (ImageField, FileField))
        ],
    )

    already_passed = already_passed.union(
        frozenset((f"{instance.__class__.__name__}:{instance.id}",))
    )
    # Go through possible relationships
    for field in chain(instance._meta.related_objects, instance._meta.concrete_fields):
        if (
            (field.one_to_one or field.many_to_one)
            and hasattr(instance, field.name)
            and (relation := getattr(instance, field.name))
        ):
            if (
                model_id := f"{relation.__class__.__name__}:{relation.id}"
            ) in already_passed:
                instance_dict[field.name] = model_id
            else:
                instance_dict[field.name] = get_instance_dict(relation, already_passed)

        if field.one_to_many or field.many_to_many:
            relations = []
            for relation in getattr(instance, field.get_accessor_name()).all():
                if (
                    model_id := f"{relation.__class__.__name__}:{relation.id}"
                ) in already_passed:
                    relations.append(model_id)
                else:
                    relations.append(get_instance_dict(relation, already_passed))
            instance_dict[field.get_accessor_name()] = relations

    return instance_dict

Koester

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Stopoff

I'd argue that the essential parts are there in text (traverse the tree, terminate recursion and reference object) but sure I can add the whole snippet here. – Koester