(Django) Trim whitespaces from charField

Question

How do I strip whitespaces (trim) from the end of a charField in Django?

Here is my Model, as you can see I've tried putting in clean methods but these never get run.

I've also tried doing name.strip(), models.charField().strip() but these do not work either.

Is there a way to force the charField to trim automatically for me?

Thanks.

from django.db import models
from django.forms import ModelForm
from django.core.exceptions import ValidationError
import datetime

class Employee(models.Model):
    """(Workers, Staff, etc)"""
    name                = models.CharField(blank=True, null=True, max_length=100)

    def save(self, *args, **kwargs):
        try:
            # This line doesn't do anything??
            #self.full_clean()
            Employee.clean(self)
        except ValidationError, e:
            print e.message_dict

        super(Employee, self).save(*args, **kwargs) # Real save

    # If I uncomment this, I get an TypeError: unsubscriptable object
    #def clean(self):
    #   return self.clean['name'].strip()

    def __unicode__(self):
        return self.name

    class Meta:
        verbose_name_plural = 'Employees'

    class Admin:pass


class EmployeeForm(ModelForm):
    class Meta:
        model = Employee

    # I have no idea if this method is being called or not  
    def full_clean(self):       
        return super(Employee), self.clean().strip()
        #return self.clean['name'].strip()

Edited: Updated code to my latest version. I am not sure what I am doing wrong as it's still not stripping the whitespace (trimming) the name field.

Oread

Answer 1

Model cleaning has to be called (it's not automatic) so place some self.full_clean() in your save method.
http://docs.djangoproject.com/en/dev/ref/models/instances/#django.db.models.Model.full_clean

As for your form, you need to return the stripped cleaned data.

return self.cleaned_data['name'].strip()

Somehow I think you just tried to do a bunch of stuff that doesn't work. Remember that forms and models are 2 very different things.

Check up on the forms docs on how to validate forms http://docs.djangoproject.com/en/dev/ref/forms/validation/

super(Employee), self.clean().strip() makes no sense at all!

Here's your code fixed:

class Employee(models.Model):
    """(Workers, Staff, etc)"""
    name = models.CharField(blank=True, null=True, max_length=100)

    def save(self, *args, **kwargs):
        self.full_clean() # performs regular validation then clean()
        super(Employee, self).save(*args, **kwargs)


    def clean(self):
        """
        Custom validation (read docs)
        PS: why do you have null=True on charfield? 
        we could avoid the check for name
        """
        if self.name: 
            self.name = self.name.strip()


class EmployeeForm(ModelForm):
    class Meta:
        model = Employee


    def clean_name(self):
        """
        If somebody enters into this form ' hello ', 
        the extra whitespace will be stripped.
        """
        return self.cleaned_data.get('name', '').strip()

Infare

I am still having problems -- it is still not stripping the whitespace (trim). I tried following the tutorial you posted, and your idea of calling full_clean() in the save method. I am struggling to understand a) Is the EmployeeForm being called? b) Is the full_clean() method being called? I have updated my code sample in my question. I am hopeful that I can get pointed in the right direction. – Oread

Thanks for your help, I'm pretty new to Django/Python; so apologies for the various mistakes and misunderstandings of how I should be implementing various features. – Oread

No problem. Just consult the docs instead of guessing and you'll figure it all out! The Django docs are godlike in clarity and examples. – Ruination

That worked great. With regards to your comment about why the name field was accepting null=True, its because I'm just starting out with Django and am not too familiar with everything. Thanks anyway though as this solved my problem. I can, hopefully, now roll it out to all textfields in the future. – Oread

null=True means the database can store a null value which is important for fields like an optional Integer field because '' would be invalid. For character fields, '' is often completely acceptable as a blank value, which also ensures that the field would always return a string, so you could safely do any python string operations on it like strip() – Ruination

Answer 2

When you're using a ModelForm instance to create/edit a model, the model's clean() method is guaranteed to be called. So, if you want to strip whitespace from a field, you just add a clean() method to your model (no need to edit the ModelForm class):

class Employee(models.Model):
    """(Workers, Staff, etc)"""
    name = models.CharField(blank=True, null=True, max_length=100)

    def clean(self):
        if self.name:
            self.name = self.name.strip()

I find the following code snippet useful- it trims the whitespace for all of the model's fields which subclass either CharField or TextField (so this also catches URLField fields) without needing to specify the fields individually:

def clean(self):
    for field in self._meta.fields:
        if isinstance(field, (models.CharField, models.TextField)):
            value = getattr(self, field.name)
            if value:
                setattr(self, field.name, value.strip())

Someone correctly pointed out that you should not be using null=True in the name declaration. Best practice is to avoid null=True for string fields, in which case the above simplifies to:

def clean(self):
    for field in self._meta.fields:
        if isinstance(field, (models.CharField, models.TextField)):
            setattr(self, field.name, getattr(self, field.name).strip())

Henninger

Good solution. However, the one thing I would change is: if value and hasattr(value, 'strip'): As some custom fields don't contain a strip method for a value (at least is my case). This protects against that. – Vercelli

Answer 3

Django 1.9 offers a simple way of accomplishing this. By using the strip argument whose default is True, you can make sure that leading and trailing whitespace is trimmed. You can only do that in form fields though in order to make sure that user input is trimmed. But that still won't protect the model itself. If you still want to do that, you can use any of the methods above.

For more information, visit https://docs.djangoproject.com/en/1.9/ref/forms/fields/#charfield

Thorax

I need to double down on this comment for future ppl that stumble upon this: after Django 1.9, even when you use ModelForm, the underneath charField will have strip to True by default. Which means you don't need to manually stipe anything. However, please note that depending on how you implemented the view, you may still see untrimmed fields because on the form object you return to the view it is still untrimmed. – Heteroplasty

After 1.9, strip is enable by default. If you want to disable this feature, check my answer: https://mcmap.net/q/572202/-django-textfield-and-charfield-is-stripping-spaces-and-blank-lines – Quizmaster

Answer 4

If you have so many data-fields to be trimmed, why not try extending CharField?

from django.db import models
from django.utils.translation import ugettext_lazy as _

class TrimCharField(models.CharField):
   description = _(
       "CharField that ignores leading" 
       " and trailing spaces in data")

   def get_prep_value(self, value)
       return trim(super(TrimCharField, self
           ).get_prep_value(value))

   def pre_save(self, model_instance, add):
       return trim(super(TrimCharField, self
           ).pre_save(model_instance, add))

UPDATE: For Django versions <= 1.7 if you want to extend field, you are to use models.SubfieldBase metaclass. So here it will be like:

class TrimCharField(six.with_metaclass(
    models.SubfieldBase, models.CharField)):

Honora

Answer 5

I'm handling this in views as a decorator. I'm also truncating field values that exceed a CharField max_length value.

from django import forms
from django import models
from django.db.models.fields import FieldDoesNotExist
from django.utils.encoding import smart_str

class CleanCharField(forms.CharField):
        """Django's default form handling drives me nuts wrt trailing
        spaces.  http://code.djangoproject.com/attachment/ticket/6362
        """
        def clean(self, value):
            if value is None:
                value = u''
            value = smart_str(value).strip()
            value = super(forms.CharField, self).clean(value)
            return value

def truncate_charfield(model):
    """decorator to truncate CharField data to model field max_length.
    Apply to the clean method in views Form:

    @truncate_charfield(MyModel)
    def clean(self):
        ...
    """
    def wrap(f):
        def wrapped_f(*args):
            f(*args)
            d = args[0].cleaned_data
            for field in model._meta.fields:
                try:
                    mf = model._meta.get_field(field.name)
                    if isinstance(mf, models.CharField) and field.name in d:
                        d[field.name] = d[field.name][:mf.max_length]
                except FieldDoesNotExist:
                    pass
            return d
        return wrapped_f
    return wrap

Toulouse

Thanks, but seems a bit too advanced for me -- I am not sure where I would put this file, how I would call it, etc; but as I pick up Django/Python I'm sure I'll pick all this up along the way. Thanks again. – Oread

zardon, you could simply put it in a module, say maxtruncate.py then import into your views.py: from maxtruncate import CleanCharField, truncate_charfield. Then replace CharField with CleanCharField and place the truncate_charfield decorator at the top of your clean method. The main advantage is you won't have your input field exceed your max_length ... which will cause a nasty runtime error from your database. – Toulouse

Answer 6

If you're still not on Django 1.9+ shame on you (and me) and drop this into your form. This is similar to @jeremy-lewis's answer but I had several problems with his.

def clean_text_fields(self):
    # TODO: Django 1.9, use on the model strip=True
    # https://docs.djangoproject.com/en/1.9/ref/forms/fields/#charfield
    from django.forms.fields import CharField
    cd = self.cleaned_data
    for field_name, field in self.fields.items():
        if isinstance(field, CharField):
            cd[field_name] = cd[field_name].strip()
            if self.fields[field_name].required and not cd[field_name]:
                self.add_error(field_name, "This is a required field.")

def clean(self):
    self.clean_text_fields()

Interclavicle