How do you put a file in a fixture in Django?

Question

I can easily fill the field of a FileField or ImageField in a Django fixture with a file name, but that file doesn't exist and when I try to test my application it fails because that file doesn't exist.

How do I correctly populate a FileField or Imagefield in a Django fixture so that the file itself is available too?

Eadith

Answer 1

I'm afraid the short answer is that you can't do this using the FileField or ImageField classes; they just store a file path and have no real concept of the file's actual data. The long answer, however, is that anything is possible if you leverage the Django API for writing your own custom model fields.

At a minimum, you'll want to implement the value_to_string method to convert the data for serialization (there's an example in the django docs at the link above). Note that the examples at the URL link above also include mention of subclassing FileField and ImageField, which is helpful for your situation!

You'll also have to decide if the data should therefore be stored in the database, or on the file system. If the former, you will have to implement your custom class as a Blob field, including customization for every DB you wish to support; you'll also have to provide some support for how the data should be returned to the user out of the database when the HTML requests a .gif/.jpg/.png/.whatever url. If the latter, which is the smarter way to go IMHO, you'll have to implement methods for serializing, de-serializing binary data to the filesystem. Either way, if you implement these as subclasses of FileField and ImageField, you should still be able to use the Admin tools and other modules that expect such django features.

If and only if you elect to use the more involved blob approach, here's a snippet of code from an old project of mind (back when I was learning Django) that handles blob for MySQL and PostgreSQL; you'll probably be able to find a number of improvements as I haven't touched it since :-) It does not handle serialization, though, so you'll have to add that using the method above.

from django.db import models
from django.conf import settings

class BlobValueWrapper(object):
    """Wrap the blob value so that we can override the unicode method.
    After the query succeeds, Django attempts to record the last query
    executed, and at that point it attempts to force the query string
    to unicode. This does not work for binary data and generates an
    uncaught exception.
    """
    def __init__(self, val):
        self.val = val

    def __str__(self):
        return 'blobdata'

    def __unicode__(self):
        return u'blobdata'


class BlobField(models.Field):
    """A field for persisting binary data in databases that we support."""
    __metaclass__ = models.SubfieldBase

    def db_type(self):
        if settings.DATABASE_ENGINE == 'mysql':
            return 'LONGBLOB'
        elif settings.DATABASE_ENGINE == 'postgresql_psycopg2':
            return 'bytea'
        else:
            raise NotImplementedError

    def to_python(self, value):
        if settings.DATABASE_ENGINE == 'postgresql_psycopg2':
            if value is None:
                return value
            return str(value)
        else:
            return value

    def get_db_prep_save(self, value):
        if value is None:
            return None
        if settings.DATABASE_ENGINE =='postgresql_psycopg2':
            return psycopg2.Binary(value)
        else:
            return BlobValueWrapper(value)

Immoralist

Do you mean that to test an application that use FielField I should create another field type and replace it? – Woodchopper

No... I read your question as meaning that you wanted fixtures to include the data. If you're just testing that FileField and ImageField work, then just test that they generate the correct URLs and paths... don't worry about the actual image data. – Immoralist

I am not testing FileField and ImageField. I'm testing a site that uses them, and one thing the sites does is crop the images and generate thumbnails. If the files are not there that code fails. And even if I work-around it, I wouldn't be testing my site completely. – Woodchopper

And why did you omit oracle backend? – Veinlet

Answer 2

There's no way to "include" the files in the serialized fixture. If creating a test fixture, you just need to do it yourself; make sure that some test files actually exist in locations referenced by the FileField/ImageField values. The values of those fields are paths relative to MEDIA_ROOT: if you need to, you can set MEDIA_ROOT in your test setUp() method in a custom test_settings.py to ensure that your test files are found wherever you put them.

EDIT: If you want to do it in your setUp() method, you can also monkeypatch default_storage directly:

from django.core.files.storage import default_storage

class MyTest(TestCase):

  def setUp(self):
    self._old_default_storage_location = default_storage.location
    default_storage.location = '/some/other/place'

  def tearDown(self):
    default_storage.location = self._old_default_storage_location

That seems to work. default_storage is a documented public API, so this should be reliable.

Daguerreotype

Setting the MEDIA_ROOT in setUp() sounds like a very good idea. I'll give it a try. Thanks. – Woodchopper

I gave this solution a try and changing settings.MEDIA_ROOT while on the test method or the setUp method doesn't change where the file is trying to be located. This solution might work, but MEDIA_ROOT should be defined somewhere else. – Woodchopper

You're right, sorry about that. settings.MEDIA_ROOT is read at startup-time by a single shared instance of FileSystemStorage that is used by all FileFields. You'd have to set it in a custom test_settings.py. – Daguerreotype