What is the clean way to unittest FileField in django?

Question

I have a model with a FileField. I want to unittest it. django test framework has great ways to manage database and emails. Is there something similar for FileFields?

How can I make sure that the unittests are not going to pollute the real application?

Thanks in advance

PS: My question is almost a duplicate of Django test FileField using test fixtures but it doesn't have an accepted answer. Just want to re-ask if something new on this topic.

Ator

Possible duplicate of Django test FileField using test fixtures – Oldfangled

Answer 1

There are several ways you could tackle this but they're all ugly since unit tests are supposed to be isolated but files are all about durable changes.

My unit tests don't run on a system with production data so it's been easy to simply reset the upload directory after each run with something like git reset --hard. This approach is in some ways the best simply because it involves no code changes and is guaranteed to work as long as you start with good test data.

If you don't actually need to do anything with that file after testing your model's save method, I'd recommend using python's excellent Mock library to completely fake the File instance (i.e. something like mock_file = Mock(spec=django.core.files.File); mock_file.read.return_value = "fake file contents") so you can completely avoid changes to your file handling logic. The Mock library has a couple of ways to globally patch Django's File class within a test method which is about as easy as this will get.

If you need to have a real file (i.e. for serving as part of a test, processing with an external script, etc.) you can use something similar to Mirko's example and create a File object after making sure it'll be stored somewhere appropriate - here are three ways to do that:

• Have your test settings.MEDIA_ROOT point to a temporary directory (see the Python tempfile module's mkdtemp function). This works fine as long as you have something like a separate STATIC_ROOT which you use for the media files which are part of your source code.
• Use a custom storage manager
• Set the file path manually on each File instance or have a custom upload_to function to point somewhere which your test setup/teardown process purges such as a test subdirectory under MEDIA_ROOT.

Edit: mock object library is new in python version 3.3. For older python versions check Michael Foord's version

Dovetailed

Thanks for this great answer. I finally choose to point to a unit-test MEDIA_ROOT by changing the value of this variable in the setUp of my test case – Ator

mock is now part of the Python standard library, available as unittest.mock in Python 3.3 onward - Gathered from REAME.rst – Visage

@Visage can you suggest an edit? That answer is definitely due for an update – Dovetailed

Answer 2

Django provides a great way to do this - use a SimpleUploadedFile or a TemporaryUploadedFile. SimpleUploadedFile is generally the simpler option if all you need to store is some sentinel data:

from django.core.files.uploadedfile import SimpleUploadedFile

my_model.file_field = SimpleUploadedFile(
    "best_file_eva.txt",
    b"these are the file contents!"   # note the b in front of the string [bytes]
)

It's one of django's magical features-that-don't-show-up-in-the-docs :). However it is referred to here and implemented here.

Limitations

Note that you can only put bytes in a SimpleUploadedFile since it's implemented using BytesIO behind the scenes. If you need more realistic, file-like behavior you can use TemporaryUploadedFile.

For Python 2

If you're stuck on python 2, skip the b prefix in the content:

my_model.file_field = SimpleUploadedFile(
    "best_file_eva.txt",
    "these are the file contents!" # no b
)

Oldfangled

I didn't know this. Thanks for your comment. I'll investigate it – Ator

The content must be str() in Python 2.x and bytes() in Python 3.x. You can't put text/unicode in there. – Kenneth

I know I'm asking this 4 years later but... Is there any way to fake the file size? – Rhinal

@Rhinal that sounds like a separate question :) and it probably depends how you are using the file size later, so include that when you open your new question – Oldfangled

This doesn't work. The assignment silently fails and the file field is still None. – Goodsized

@Goodsized Did you save afterwards? – Androgynous

source: github.com/django/django/blob/master/django/core/files/… – Igorot

This worked for me but the file is being saved on disk which I did not want. I resolved this by mocking the file storage backend as stated in this article cscheng.info/2018/08/21/… – Assuage

Answer 3

I normally test filefields in models using doctest

>>> from django.core.files import File
>>> s = SimpleModel()
>>> s.audio_file = File(open("media/testfiles/testaudio.wav"))
>>> s.save()
>>> ...
>>> s.delete()

If I need to I also test file uploads with test clients.

As for fixtures, I simply copy the files i need in a test folder, after modifying the paths in the fixture.

e.g.

In a fixture containing models with filefiels pointing to a directory named "audio", you replace "audio": "audio/audio.wav" with "audio": "audio/test/audio.wav" .
Now all you have to do is copy the test folder, with the necessary files, in "audio" in the test setUp and then delete it in tearDown.

Not the cleanest way ever i think, but that's what i do.

Climate

Do I need to close the file handle manually to free the resources used by it? – Kitten

Answer 4

If you just want to create an object that requires FileField and don't want to use this field then you can just pass any (existing or not) relative path like this:

example_object = models.ExampleModel({'file': "foo.bar"})
example_object.save()

Then it's ready for use.

Androgynous

Answer 5

I guess that easiest way is to use ContentFile class:

file = ContentFile('text', 'name')
my_model = MyModel()
my_model.file = file
my_model.save()

Grozny

Answer 6

There is some way to mock storage.

After reading django source code, the below is my implementation to avoid uploading file to s3 regardless of using default storage or assigning the storage in filefield or imagefield.

from unittest.mock import patch
from django.db.models.fields.files import FileField, FieldFile, ImageField, ImageFieldFile
from django.core.files.storage import FileSystemStorage
from rest_framework.test import APITestCase


class CustomFileSystemStorage(FileSystemStorage):
    def url(self, *args, **kwargs):
        return self.path(*args, **kwargs)


class CustomFieldFile(FieldFile):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.storage = CustomFileSystemStorage()
        
        
class CustomImageFieldFile(ImageFieldFile):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.storage = CustomFileSystemStorage()


class PatchMeta(type):
    """A metaclass to patch all inherited classes."""

    def __new__(meta, name, bases, attrs):
        cls = type.__new__(meta, name, bases, attrs)
        cls = patch.object(ImageField, 'attr_class', CustomImageFieldFile)(cls)
        cls = patch.object(FileField, 'attr_class', CustomFieldFile)(cls)
        return cls


class CustomBaseTestCase(APITestCase, metaclass=PatchMeta):
    """You can inherit this class to do any testcase for mocking storage"""
    pass

With inheriting CustomBaseTestCase class, you can change your storage to avoid uploading files to any server

Jimenez