Django File upload size limit

Question

I have a form in my django app where users can upload files.
How can i set a limit to the uploaded file size so that if a user uploads a file larger than my limit the form won't be valid and it will throw an error?

Prognostication

Similar question with answer: #2895414 – Atalanta

@DaveGallagher: Using a upload handler does not present the user with a pretty error message, it just drops the connection. – Fettling

Possible duplicate of Max image size on file upload – Decastere

Answer 1

This code might help:

# Add to your settings file
CONTENT_TYPES = ['image', 'video']
# 2.5MB - 2621440
# 5MB - 5242880
# 10MB - 10485760
# 20MB - 20971520
# 50MB - 5242880
# 100MB 104857600
# 250MB - 214958080
# 500MB - 429916160
MAX_UPLOAD_SIZE = 52428800

#Add to a form containing a FileField and change the field names accordingly.
from django.template.defaultfilters import filesizeformat
from django.utils.translation import ugettext_lazy as _
from django.conf import settings
def clean_content(self):
    content = self.cleaned_data['content']
    content_type = content.content_type.split('/')[0]
    if content_type in settings.CONTENT_TYPES:
        if content._size > settings.MAX_UPLOAD_SIZE:
            raise forms.ValidationError(_('Please keep filesize under %s. Current filesize %s') % (filesizeformat(settings.MAX_UPLOAD_SIZE), filesizeformat(content._size)))
    else:
        raise forms.ValidationError(_('File type is not supported'))
    return content

Taken from: Django Snippets - Validate by file content type and size

Deas

Do you know why such values are used, look like 10 * (some power of 2)? – Vuillard

@AnthonyLozano Actually, MiB is always always always 1048576 bytes, it is not a power of ten. As for MB, it is ambiguous, it can either mean 1000000 bytes if you're following IEC standards, or 1048576 bytes if you're using Windows and the like. The Wikipedia article you linked to is proof. – Ruction

Pretty sure you lost a 0 at the end of "5242880". Should be "52428800" – Enfold

In Django 1.10 use content.size (no underscore) – Allowedly

Do not set MAX_UPLOAD_SIZE to a string. It should be a number -- this code will allow any size upload since the first ValidationError can't be reached. – Sturgis

what if I am not using a Django form or FileField and just have a regular HTML form that POST's the file to a Django view? – Powel

I edited the answer to fix MAX_UPLOAD_SIZE type and value. – Fives

Answer 2

You can use this snippet formatChecker. What it does is

• it lets you specify what file formats are allowed to be uploaded.

• and lets you set the limit of file size of the file to be uploaded.

First. Create a file named formatChecker.py inside the app where the you have the model that has the FileField that you want to accept a certain file type.

This is your formatChecker.py:

from django.db.models import FileField
from django.forms import forms
from django.template.defaultfilters import filesizeformat
from django.utils.translation import ugettext_lazy as _

class ContentTypeRestrictedFileField(FileField):
    """
    Same as FileField, but you can specify:
        * content_types - list containing allowed content_types. Example: ['application/pdf', 'image/jpeg']
        * max_upload_size - a number indicating the maximum file size allowed for upload.
            2.5MB - 2621440
            5MB - 5242880
            10MB - 10485760
            20MB - 20971520
            50MB - 5242880
            100MB - 104857600
            250MB - 214958080
            500MB - 429916160
    """
    def __init__(self, *args, **kwargs):
        self.content_types = kwargs.pop("content_types", [])
        self.max_upload_size = kwargs.pop("max_upload_size", 0)

        super(ContentTypeRestrictedFileField, self).__init__(*args, **kwargs)

    def clean(self, *args, **kwargs):
        data = super(ContentTypeRestrictedFileField, self).clean(*args, **kwargs)

        file = data.file
        try:
            content_type = file.content_type
            if content_type in self.content_types:
                if file._size > self.max_upload_size:
                    raise forms.ValidationError(_('Please keep filesize under %s. Current filesize %s') % (filesizeformat(self.max_upload_size), filesizeformat(file._size)))
            else:
                raise forms.ValidationError(_('Filetype not supported.'))
        except AttributeError:
            pass

        return data

Second. In your models.py, add this:

from formatChecker import ContentTypeRestrictedFileField

Then instead of using 'FileField', use this 'ContentTypeRestrictedFileField'.

Example:

class Stuff(models.Model):
    title = models.CharField(max_length=245)
    handout = ContentTypeRestrictedFileField(upload_to='uploads/', content_types=['video/x-msvideo', 'application/pdf', 'video/mp4', 'audio/mpeg', ],max_upload_size=5242880,blank=True, null=True)

You can change the value of 'max_upload_size' to the limit of file size that you want. You can also change the values inside the list of 'content_types' to the file types that you want to accept.

Vaporescence

what a great underrated answer! More complete and slightly better than the validated one. – Vilberg

gives an error __init__() got an unexpected keyword argument content_types while creating a database – Leavelle

Indentation is wrong in the class above, that's why it fails – Calfskin

line 23, in init self.content_types = kwargs.pop("content_types") KeyError: 'content_types' - keeps giving me this error – Calypso

Nitpick: using MiB units are preferable because 1MiB always equals 1048576 bytes, where 1MB could equal 1048576 bytes or 1000000 bytes. – Ruction

This really should be built into Django. – Quintie

Even with the indentation fix, I still get the 'content_types' error. – Quintie

Oh I see - after changing the field type, content_types and max_upload_size become required arguments on each field that use them. – Quintie

This does not check the size nor content type for me, the clean method is even not called at all. Do I need a special form for this? – Cornwall

As @Calypso said. I also got KeyError by using this code – Rozier

Where is the package containing this ? – Decastere

Those who encounters this should remember from documentation kwargs.pop should have a default. Change to this self.content_types = kwargs.pop("content_types", []) self.max_upload_size = kwargs.pop("max_upload_size", []) – Destalinization

Answer 3

another solution is using validators

from django.core.exceptions import ValidationError

def file_size(value): # add this to some file where you can import it from
    limit = 2 * 1024 * 1024
    if value.size > limit:
        raise ValidationError('File too large. Size should not exceed 2 MiB.')

then in your form with the File field you have something like this

image = forms.FileField(required=False, validators=[file_size])

Selden

This is my favourite, as the others access a private variable _size and this one doesn't. – Ruction

Careful! Validations are only called when you are using a Form to save data, Else you have to manually call validations eg. 'instance.full_clean()' before saving to db. – Geoponic

@Geoponic I believe the question does indicate that the file is being received via Forms, so no worries. – Selden

this didnt work for me, validator isn't being called – Mcneil

@ifedapoolarewaju will this work if the user has to upload more than one file ? – Braden

@ifedapoolarewaju Hi, I'm interested in using this as server-side file size validation, but in my case, files over 2mb are still uploaded and no validation error is thrown when addingfrom django.core.exceptions import ValidationError... to forms.py and calling validators=[file_size] on forms.FileField. Would this be an issue with my views.py? – Autoharp

@Autoharp I'm not sure, it could be anything. Try modifying the file_size function to print(value.size, 'file size received') and then inspect the console output to be certain what file size value is being received by the validator – Selden

@ifedapoolarewaju Thanks, with from django.core.exceptions import ValidationError and def file_size.... added to the top of forms.py, and your new file_size function, I get the error name 'value' is not defined pointing to the line with the new file_size function. – Autoharp

@Autoharp it seems you are printing outside the body of the function. Can you print it right above the if value.size > limit: line? – Selden

Answer 4

I believe that django form receives file only after it was uploaded completely.That's why if somebody uploads 2Gb file, you're much better off with web-server checking for size on-the-fly.

See this mail thread for more info.

Neomaneomah

I agree with you on this but in my case i need the limit to be in Django. – Prognostication

At the time of writing (2 years ago), django would simply DoS with heavy file upload. Right now things are different, and depending on the purpose of the restriction it could go either way – Neomaneomah

Answer 5

Server side

My favourite method of checking whether a file is too big server-side is ifedapo olarewaju's answer using a validator.

Client side

The problem with only having server-side validation is that the validation only happens after the upload is complete. Imagine, uploading a huge file, waiting for ages, only to be told afterwards that the file is too big. Wouldn't it be nicer if the browser could let me know beforehand that the file is too big?

Well, there is a way to this client side, using HTML5 File API!

Here's the required Javascript (depending on JQuery):

$("form").submit(function() {
  if (window.File && window.FileReader && window.FileList && window.Blob) {
    var file = $('#id_file')[0].files[0];

    if (file && file.size > 2 * 1024 * 1024) {
      alert("File " + file.name + " of type " + file.type + " is too big");
      return false;
    }
  }
});

Of course, you still need server-side validation, to protect against malicious input, and users that don't have Javascript enabled.

Ruction

Thanks for the addition, it makes sense to do it both ways. The above works for me. – Zolazoldi

Answer 6

Just a short note on the snippet that was included in this thread:

Take a look at this snippet: http://www.djangosnippets.org/snippets/1303/

It was very usefull, however it's including a few minor mistakes. More robust code should look like this:

# Add to your settings file
CONTENT_TYPES = ['image', 'video']
# 2.5MB - 2621440
# 5MB - 5242880
# 10MB - 10485760
# 20MB - 20971520
# 50MB - 5242880
# 100MB - 104857600
# 250MB - 214958080
# 500MB - 429916160
MAX_UPLOAD_SIZE = "5242880"

#Add to a form containing a FileField and change the field names accordingly.
from django.template.defaultfilters import filesizeformat
from django.utils.translation import ugettext_lazy as _
from django.conf import settings
def clean_content(self):
    if content != None:
        content = self.cleaned_data['content']
        content_type = content.content_type.split('/')[0]
        if content_type in settings.CONTENT_TYPES:
            if content._size > int(settings.MAX_UPLOAD_SIZE):
                raise forms.ValidationError(_(u'Please keep filesize under %s. Current filesize %s') % (filesizeformat(settings.MAX_UPLOAD_SIZE), filesizeformat(content._size)))
        else:
            raise forms.ValidationError(_(u'File type is not supported'))
        return content

There are just a few improvements:

First of all I'm detecting if the file field is empty (None) - without it, Django will cast an exception in web browser.

Next is type casting in int(settings.MAX_UPLOAD_SIZE), because that setting value is a string. Strings cannot be used for comparing with numbers.

Last but not least, the unicode 'u' prefix in ValidationError function.

Thank you very much for this snippet!

Torchwood

I'm using a similar method, just using python-magic instead of reading django content_type field, but I faced an issue. I accept pdf files (mime type 'application/pdf'). The problem is that sometimes the mime type seems to be "application/octet-stream" even for pdf files. I don't want to add that mime type to my accepted types list, since otherwise also other document types would be accepted (es. excel). Does someone know how to fix this issue? – Unfaithful

Try to use file extensions check for '.pdf' instead of content_type checking. This is simpler and more robust at the same time. – Antecedency

Answer 7

In my case, django limit the upload file size. Add the following settings will remove restriction.

# allow upload big file
DATA_UPLOAD_MAX_MEMORY_SIZE = 1024 * 1024 * 15  # 15M
FILE_UPLOAD_MAX_MEMORY_SIZE = DATA_UPLOAD_MAX_MEMORY_SIZE

Raddy

Whenever I try to upload large images in my Django app it doesn't deliver them to the server. Thank you so much for this help. – Largess

is FILE_UPLOAD_MAX_MEMORY_SIZE needed to be the same? docs say it would start streaming to the filesystem, starting from this limit on...so...only DATA_UPLOAD_MAX_MEMORY_SIZE does limit maximum uploaded filesize. – Sustain

also, I prefer this answer - oneliner in settings, done...if you dont need extensive different limits, this should be the preferred solution. – Sustain

@Raddy You saved my day! in addition to this, I have added client_max_body_size 25M; in Nginx config – Decarburize

Answer 8

If someone is looking for a form FileField variant of @angelo solution then here it is

from django import forms
from django.template.defaultfilters import filesizeformat
from django.utils.translation import ugettext_lazy as _
from django.core.exceptions import ValidationError

class RestrictedFileField(forms.FileField):
    """
    Same as FileField, but you can specify:
    * content_types - list containing allowed content_types. Example: ['application/pdf', 'image/jpeg']
    * max_upload_size - a number indicating the maximum file size allowed for upload.
        2.5MB - 2621440
        5MB - 5242880
        10MB - 10485760
        20MB - 20971520
        50MB - 5242880
        100MB - 104857600
        250MB - 214958080
        500MB - 429916160
"""

    def __init__(self, *args, **kwargs):
        self.content_types = kwargs.pop("content_types")
        self.max_upload_size = kwargs.pop("max_upload_size")

        super(RestrictedFileField, self).__init__(*args, **kwargs)

    def clean(self, data, initial=None):
        file = super(RestrictedFileField, self).clean(data, initial)

        try:
            content_type = file.content_type
            if content_type in self.content_types:
                if file._size > self.max_upload_size:
                    raise ValidationError(_('Please keep filesize under %s. Current filesize %s') % (
                        filesizeformat(self.max_upload_size), filesizeformat(file._size)))
            else:
                raise ValidationError(_('Filetype not supported.'))
        except AttributeError:
            pass

        return data

Then create a form as

class ImageUploadForm(forms.Form):
    """Image upload form."""
    db_image = RestrictedFileField(content_types=['image/png', 'image/jpeg'],
                                   max_upload_size=5242880)

Geoponic

Answer 9

Another elegant solution with validators that does not hard-code the max file size is by using a class based validator:

from django.core.exceptions import ValidationError
from django.core.validators import MaxValueValidator
from django.utils.translation import ugettext as _

class MaxSizeValidator(MaxValueValidator):
    message = _('The file exceed the maximum size of %(limit_value)s MB.')

    def __call__(self, value):
        # get the file size as cleaned value
        cleaned = self.clean(value.size)
        params = {'limit_value': self.limit_value, 'show_value': cleaned, 'value': value}
        if self.compare(cleaned, self.limit_value * 1024 * 1024): # convert limit_value from MB to Bytes
            raise ValidationError(self.message, code=self.code, params=params)

and then, in your model, for example:

image = models.ImageField(verbose_name='Image', upload_to='images/', validators=[MaxSizeValidator(1)])

EDIT: here is the source code of MaxValueValidator for more details on this works.

Colossae

Answer 10

I want to thank all the folks who have provided various different solutions to this problem. I had additional requirements where I wanted to (a) do file length validation in JavaScript before submission, (b) do a second line of defense in-server validation in the forms.py, (c) keep all hard-coded bits including end-user messages in forms.py, (d) I wanted my views.py have as little file-related code as possible, and (d) upload the file information to my database since these are small files that I want to only serve to logged in users and instantly delete when the Meal model items are deleted (i.e. so just dropping them in /media/ is not sufficient).

First the model:

class Meal(models.Model) :
    title = models.CharField(max_length=200)
    text = models.TextField()

    # Picture (you need content type to serve it properly)
    picture = models.BinaryField(null=True, editable=True)
    content_type = models.CharField(max_length=256, null=True, help_text='The MIMEType of the file')

    # Shows up in the admin list
    def __str__(self):
        return self.title

Then you need a form that both does the in-server validation and the pre-save conversion from InMemoryUploadedFile to bytes and grabbing the Content-Type for later serving.

class CreateForm(forms.ModelForm):
    max_upload_limit = 2 * 1024 * 1024
    max_upload_limit_text = str(max_upload_limit) # A more natural size would be nice
    upload_field_name = 'picture'
    # Call this 'picture' so it gets copied from the form to the in-memory model
    picture = forms.FileField(required=False, label='File to Upload <= '+max_upload_limit_text)

    class Meta:
        model = Meal
        fields = ['title', 'text', 'picture']

    def clean(self) :  # Reject if the file is too large
        cleaned_data = super().clean()
        pic = cleaned_data.get('picture')
        if pic is None : return
        if len(pic) > self.max_upload_limit:
            self.add_error('picture', "File must be < "+self.max_upload_limit_text+" bytes")

    def save(self, commit=True) : # Convert uploaded files to bytes
        instance = super(CreateForm, self).save(commit=False)
        f = instance.picture   # Make a copy
        if isinstance(f, InMemoryUploadedFile):
            bytearr = f.read();
            instance.content_type = f.content_type
            instance.picture = bytearr  # Overwrite with the actual image data

        if commit:
            instance.save()
        return instance

In the template, add this code (adapted from a previous answer):

<script>
$("#upload_form").submit(function() {
  if (window.File && window.FileReader && window.FileList && window.Blob) {
      var file = $('#id_{{ form.upload_field_name }}')[0].files[0];
      if (file && file.size > {{ form.max_upload_limit }} ) {
          alert("File " + file.name + " of type " + file.type + " must be < {{ form.max_upload_limit_text }}");
      return false;
    }
  }
});
</script>

Lately I am moving away from JQuery where I can, so the above client side code to check file size in Vanilla JS is:

<script>
  document.getElementById("upload_form").addEventListener("submit", function() {
    console.log('Checking file size');
    if (window.File && window.FileReader && window.FileList && window.Blob) {
        var file = document.getElementById('id_{{ form.upload_field_name }}').files[0];
        if (file && file.size > {{ form.max_upload_limit }} ) {
            alert("File " + file.name + " of type " + file.type + " must be < {{ form.max_upload_limit_text }}");
        return false;
      }
    }
  });
</script>

Here is the view code that handles both Create and Update:

class MealFormView(LoginRequiredMixin, View):
    template = 'meal_form.html'
    success_url = reverse_lazy('meals')
    def get(self, request, pk=None) :
        if not pk :
            form = CreateForm()
        else:
            meal = get_object_or_404(Meal, id=pk, owner=self.request.user)
            form = CreateForm(instance=meal)
        ctx = { 'form': form }
        return render(request, self.template, ctx)

    def post(self, request, pk=None) :
        if not pk:
            form = CreateForm(request.POST, request.FILES or None)
        else:
            meal = get_object_or_404(Meal, id=pk, owner=self.request.user)
            form = CreateForm(request.POST, request.FILES or None, instance=meal)

        if not form.is_valid() :
            ctx = {'form' : form}
            return render(request, self.template, ctx)

        form.save()
        return redirect(self.success_url)

This is a very simple view that makes sure that request.FILES is passed in during the creation of the instance. You could almost use the generic CreateView if it would (a) use my form and (b) pass request.files when making the model instance.

Just to complete the effort, I have the following simple view to stream the file:

def stream_file(request, pk) :
    meal = get_object_or_404(Meal, id=pk)
    response = HttpResponse()
    response['Content-Type'] = meal.content_type
    response['Content-Length'] = len(meal.picture)
    response.write(meal.picture)
    return response

This does not force users to be logged in, but I omitted that since this answer is already too long.

Custom

do not use binary fields to store image data – Alvin

I agree but ... My example is for a simple case and as a student demonstration project so it is not worth setting up a disk based blob store. The default approach to files in Django is fine for a blog or other "public" content - because it does not have any access controls when viewing files. My applications are education oriented where you must protect the privacy of things like student avatar images. The default Django approach does not include the ability to keep images private - my approach above does. – Custom

Answer 11

from django.forms.utils import ErrorList

class Mymodelform(forms.ModelForm):
    class Meta:
        model = Mymodel
        fields = '__all__'

    def clean(self):image = self.cleaned_data.get('image')
        # 5MB - 5242880
        if org_image._size > 5242880:            
            self._errors["image"] = ErrorList([u"Image too heavy."])

Hundredweight

Answer 12

You can extend Django's MaxValueValidator and overwrite it's clean() to return the file size:

from django.core.validators import MaxValueValidator
from django.utils.deconstruct import deconstructible
from django.utils.translation import ugettext_lazy as _


@deconstructible
class MaxKibFileSizeValidator(MaxValueValidator):
    message = _('File size %(show_value)d KiB exceeds maximum file size of %(limit_value)d KiB.')

    def clean(self, filefield) -> float:
        return filefield.file.size / 1024

Fled

Answer 13

I struggled with limiting both the file type and size of uploaded documents. So here is my final solution (partially based on one of the solutions above):

• I created a filechecker.py and added :

import os
from django.core.exceptions import ValidationError

def file_size(value):
    limit = 1048576
    if value.size > limit:
        raise ValidationError('File too large. Size should not exceed 1 MB.')
    
def file_extension(value):
    ext = os.path.splitext(value.name)[1]
    valid_extensions = ['.pdf', '.doc', '.docx']
    if not ext in valid_extensions:
        raise ValidationError('Unsupported file type. Only Pdf and MsWord files are allowed.')

• Imported the functions in forms.py and used them in the list of validators for my cv field:

# ....other imports
from .filechecker import file_size, file_extension

class JobApplicationForm(forms.ModelForm):
    cv = forms.FileField(required=True, validators=[file_size, file_extension] )
    class Meta:
        model = jobApplicant
        fields = ["email", "cv"]

Gillis