Rails before_validation strip whitespace best practices

Question

I would like my User model to sanitize some input before before save. For now some simple whitespace stripping will do. So to avoid people registering with "Harry " and pretend to be "Harry", for example.

I assume it is a good idea to do this stripping before validation, so that the validates_uniqueness_of can avoid accidental duplicates.

class User < ActiveRecord::Base
  has_many :open_ids

  validates_presence_of :name
  validates_presence_of :email
  validates_uniqueness_of :name
  validates_uniqueness_of :email
  validates_format_of :email, :with => /\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\Z/i

  before_validation :strip_whitespace, :only => [:name, :email, :nick]

  private
  def strip_whitespace(value)
    value.responds_to?('strip') ? value.strip : value
  end
end

However, this code comes with an error ArgumentError: wrong number of arguments (0 for 1). I assumed the callback would be passed the values.

Also: is this stripping actually a good idea? Or should I rather validate on space and tell the user that "Harry " contains invalid spacess (I want to allow "Harry Potter" but not "Harry\s\sPotter").

Edit: As pointed out in a comment, my code is wrong (which is why I was asking the question a.o.). Please make sure you read the accepted answer in addition to my question for the correct code and to avoid the same mistakes I made.

Tarantella

For others stumbling upon this - before_validation does not have an :only option. And the callback does not take an argument. See answers for more details. – Taxi

Answer 1

I don't believe before_validation works like that. You probably want to write your method like this instead:

def strip_whitespace
  self.name = self.name.strip unless self.name.nil?
  self.email = self.email.strip unless self.email.nil?
  self.nick = self.nick.strip unless self.nick.nil?
end

You could make it more dynamic if you want using something like self.columns, but that's the gist of it.

Rosalia

I added an unless self.name.blank? behind them, to avoid stripping NIL-values. – Tarantella

Depending on your class you can consider ruby def strip_whitespace self.email = email.strip end – Kellie

@Tarantella - I'd be better to add if self.name.respond_to?(:strip). – Vitreous

strip does not handle Unicode space characters. See bugs.ruby-lang.org/issues/7845. As of Ruby 2.3.0 the issue is still present. – Toots

I realize this is very old, but I wanted to point out two things.. First, instead of self.name = self.name.strip unless self.name.nil?, I've come to prefer self.name.try(&:strip!). But if you are really looking to remove whitespace from the beginning and end, I find self.name.gsub! /(\A\s*|\s*\z)/, '' to be most reliable. – Loralorain

@Loralorain why do not you post your answer with some explanations? – Aenneea

This answer is fine but I think this answer is even better. – Isaacisaacs

Answer 2

There are several gems to do this automatically. Those gems work in the similar way of creating callback in before_validation. One good gem is at https://github.com/holli/auto_strip_attributes

gem "auto_strip_attributes", "~> 2.2"

class User < ActiveRecord::Base
  auto_strip_attributes :name, :nick, nullify: false, squish: true
  auto_strip_attributes :email
end

Stripping is often a good idea. Especially for leading and trailing whitespaces. User often creates trailing spaces when copy/pasting value to a form. With names and other identifying strings you also might want squish the string. So that "Harry    Potter" will become "Harry Potter" (squish option in the gem).

Densitometer

Answer 3

Charlie's answer is good, but there's a little verbosity. Here's a tighter version:

def clean_data
  # trim whitespace from beginning and end of string attributes
  attribute_names.each do |name|
    if send(name).respond_to?(:strip)
      send("#{name}=", send(name).strip)
    end
  end
end

The reason we use

self.foo = "bar"

instead of

foo = "bar"

in the context of ActiveRecord objects is that Ruby interprets the latter as a local variable assignment. It will just set the foo variable in your method scope, instead of calling the "foo=" method of your object.

But if you are calling a method, there is no ambiguity. The interpreter knows you're not referring to a local variable called foo because there is none. So for example with:

self.foo = foo + 1

you need to use "self" for the assignment, but not to read the current value.

Corporal

I'm using this but with changed.each instead of attributes_names to limit it to the fields that were changed. – Clearsighted

Answer 4

I'd like to add one pitfall that you might experience with the "before_validations" solutions above. Take this example:

u = User.new(name: " lala")
u.name # => " lala"
u.save
u.name # => "lala"

This means you have an inconsistent behavior based on whether your object was saved or not. If you want to address this, I suggest another solution to your problem: overwriting the corresponding setter methods.

class User < ActiveRecord::Base
  def name=(name)
    write_attribute(:name, name.try(:strip))
  end
end

I also like this approach because it does not force you to enable stripping for all attributes that support it - unlike the attribute_names.each mentioned earlier. Also, no callbacks required.

Bareheaded

Thanks for your comment Ben. I'm using the abovementioned approach in Rails 3 without any problems. Also, it still is the approach mentioned in the docs for 3.2.8: api.rubyonrails.org/classes/ActiveRecord/Base.html. Did you experience any issues with it? – Bareheaded

Answer 5

Instead we can write a better method more generic regardless whatever may be the type of attributes with the object(might have 3 string type fields, few booleans, few numeric)

before_validation :strip_input_fields


def strip_input_fields
  self.attributes.each do |key, value|
    self[key] = value.strip if value.respond_to?("strip")
  end
end

Hope that will helps someone!

Ablaut

respond_to?() is a charm!!! I'm using squish instead of strip, because it also transforms few spaces in one inside of the string – Belligerency

Answer 6

StripAttributes Gem

I used strip_attributes. It's really awesome and easy to implement.

Default Behavior

class DrunkPokerPlayer < ActiveRecord::Base
  strip_attributes
end

By default, this will only strip the leading and trailing whitespaces and will act on all attributes of the model. This is ideal because it's not destructive and doesn't require you to specify which attributes need to be striped.

Using except

# all attributes will be stripped except :boxers
class SoberPokerPlayer < ActiveRecord::Base
  strip_attributes :except => :boxers
end

Using only

# only :shoe, :sock, and :glove attributes will be stripped
class ConservativePokerPlayer < ActiveRecord::Base
  strip_attributes :only => [:shoe, :sock, :glove]
end

Using allow_empty

# Empty attributes will not be converted to nil
class BrokePokerPlayer < ActiveRecord::Base
  strip_attributes :allow_empty => true
end

Using collapse_spaces

# Sequential spaces in attributes will be collapsed to one space
class EloquentPokerPlayer < ActiveRecord::Base
  strip_attributes :collapse_spaces => true
end

Using regex

class User < ActiveRecord::Base
  # Strip off characters defined by RegEx
  strip_attributes :only => [:first_name, :last_name], :regex => /[^[:alpha:]\s]/
  # Strip off non-integers
  strip_attributes :only => [:phone], :regex => /[^0-9]/
end

Transversal

Thanks for the link to that gem, but seems it only works up to Rails 3.2 – Hemorrhoidectomy

@Hemorrhoidectomy From the current .gemspec, it looks like it works from Rails 3 - Rails 5: "activemodel", ">= 3.0", "< 6.0" – Panarabism

@JoshuaPinter Thanks for the info, I guess it was updated. – Hemorrhoidectomy

Answer 7

Starting with Ruby 2.3.0 you can use the Safe Navigation Operator(&.)

before_validation :strip_whitespace

def strip_whitespace
  self.name&.strip!
  self.email&.strip!
  self.nick&.strip!
end

GEMS:
https://github.com/rmm5t/strip_attributes/
https://github.com/holli/auto_strip_attributes/

Serriform

Answer 8

I like Karl's answer, but is there a way to do it without referencing each of the attributes by name? That is, is there a way to just run through the model attributes and call strip on each one (if it responds to that method)?

This would be desirable so I don't have to update the remove_whitespace method whenever I change the model.

UPDATE

I see that Karl implied that you might want to do this sort of thing. I didn't immediately know how it could be done, but here's something that works for me as described above. There' probably a better way to do it, but this works:

def clean_data
  # trim whitespace from beginning and end of string attributes
  attribute_names().each do |name|
  if self.send(name.to_sym).respond_to?(:strip)
    self.send("#{name}=".to_sym, self.send(name).strip)
  end
end

end

Sugared

that looks like a superior solution, and worked great, thanks – Shelter

Great solution. But it can be optimized more: instead of attributes_names method we can use changes.keys so second time only changed attributes can be stripped. – Amoreta

Answer 9

If you have access to ActiveSupport, use squish instead of strip.

http://api.rubyonrails.org/classes/String.html#method-i-squish

Divisibility

Damn, I'd never noticed this method before. A useful one! Although it's worth noting that you might not always want to use squish instead of strip, e.g. for a long piece of text like a blog post you probably want to preserve the user's inner whitespace. – Tibetoburman

Careful with this as it will remove any multiple spaces and remove any newlines as well. Certainly not what you want most of the time when accepting user input and a lot more invasive than just stripping the leading and trailing whitespaces. – Panarabism

Answer 10

Ruby on Rails 7.1 introduced ActiveRecord::Base::normalizes which allows defining data sanitization callbacks like this:

normalizes :name, :email, :nick, with: ->(value) { value.strip }

This method is able to handle nil values and will per default not call the normalization lambda when the value is nil. That means no extra check for nil required.

When the normalization should be called for nil values too, then you can set apply_to_nil: true, to do stuff like this

normalize :name, apply_to_nil: true, with: ->(name) { name&.strip || 'NN' }

which would normalize the name attribute by stripping whitespace if a name is set, otherwise it would set the name to the string "NN"

Nap

Answer 11

Overriding the attribute write methods is another good way. For example:

class MyModel
  def email=(value)
    super(value.try(:strip))
  end
end

Then any part of the application that sets the value will have it stripped, including assign_attributes and so on.

Rodgers

Answer 12

Here's an alternative approach, if you are mostly concerned with users mis-entering data in your front-end forms...

# app/assets/javascripts/trim.inputs.js.coffee
$(document).on "change", "input", ->
  $(this).val $(this).val().trim()

Then include the file in your application.js if you aren't already including the whole tree.

This will ensure that every input gets leading & trailing whitespace removed before it is submitted to be saved by Rails. It's bound on document, and delegated to inputs, so any inputs added to the page later will be processed as well.

Pros:

• Does not require listing individual attributes by name
• Does not require any metaprogramming
• Does not require external library dependencies

Cons:

• Data submitted any other way than the forms (eg, via API) will not be trimmed
• Does not have advanced features like squish (but you could add that yourself)
• As mentioned in comments, does not work if JS is disabled (but who codes for that?)

Spermicide

Nice addition. Thanks. Though this will also catch passwords, where people might deliberately add a space. After which that space gets magically removed. IMHO any password-field should be excluded from the JS-trimming. – Tarantella

Regardless what validations/clean up are present on client side, backend has to do its own. – Toots

Answer 13

Since I can't comment yet, I'll have to ask here: which method is giving the ArgumentError? strip, or responds_to?

Also, .strip removes only leading and trailing whitespace. If you want "Harry Potter" with two spaces to not be accepted, you would either have to use a regex or, more simply, you could call .split, which removes spaces, and re-concatenate the string with a single space.

As far as if stripping is a good idea, I don't see a problem when it is just the leading/trailing whitespace. If there are multiple spaces in between words though, I would notify the user instead of automatically removing the extra spaces and giving the user a login that is not what they submitted.

Lashing

My guess is strip_whitespace is throwing the error. You don't pass along values to the validation callbacks because the record gets passed. I don't believe you can do the :only style, either. – Pizza

About the two-spaces-in-the-middle: yes, that is what I will add later in validations. About the method that throws the error: that is strip_whitespace itself. – Tarantella

Answer 14

Another gem option is attribute_normalizer:

# By default it will strip leading and trailing whitespace
# and set to nil if blank.
normalize_attributes :author, :publisher

:strip Will strip leading and trailing whitespace.

normalize_attribute  :author, :with => :strip

Sporogenesis

Answer 15

A better alternative is to overwrite the setter method and use value.squish. Its cleaner and you don't have to use before_validation:

class User < ActiveRecord::Base
  def name=(value)
    super(value.squish)
  end  
end

Arias

Answer 16

Rails 7.1 introduced ActiveRecord::Base::normalizes

class User < ApplicationRecord
  normalizes :email, with: ->(email) { email.strip.downcase }
end

It invokes before validation

It is safe because doesn't (but optionally can) apply to nil

User.create(email: " [email protected] \n")
# => #<User email: "[email protected]">

User.find_by(email: "\[email protected] \t")
# => #<User email: "[email protected]">

If you have legacy non-normalized record, you can normalize it with ActiveRecord::Base#normalize_attribute

user.email # => "[email protected]  "
user.normalize_attribute(:email)
user.email # => "[email protected]"
user.save

Scalf