Existing data serialized as hash produces error when upgrading to Rails 5

Question

I am currently upgrading a Ruby on Rails app from 4.2 to 5.0 and am running into a roadblock concerning fields that store data as a serialized hash. For instance, I have

class Club
  serialize :social_media, Hash
end

When creating new clubs and inputting the social media everything works fine, but for the existing social media data I'm getting:

ActiveRecord::SerializationTypeMismatch: Attribute was supposed to be a Hash, but was a ActionController::Parameters.

How can I convert all of the existing data from ActionController::Parameter objects to simple hashes? Database is mysql.

Olivette

Which database are you using? I ask mostly because there might be better options that serialize and since you're going to be fixing and rewriting all this data anyway, this might be a good time to get rid of serialize completely. – Frazzle

Currently using myql. – Olivette

Answer 1

From the fine manual:

serialize(attr_name, class_name_or_coder = Object)

[...] If class_name is specified, the serialized object must be of that class on assignment and retrieval. Otherwise SerializationTypeMismatch will be raised.

So when you say this:

serialize :social_media, Hash

ActiveRecord will require the unserialized social_media to be a Hash. However, as noted by vnbrs, ActionController::Parameters no longer subclasses Hash like it used to and you have a table full of serialized ActionController::Parameters instances. If you look at the raw YAML data in your social_media column, you'll see a bunch of strings like:

--- !ruby/object:ActionController::Parameters...

rather than Hashes like this:

---\n:key: value...

You should fix up all your existing data to have YAMLized Hashes in social_media rather than ActionController::Parameters and whatever else is in there. This process will be somewhat unpleasant:

1. Pull each social_media out of the table as a string.
2. Unpack that YAML string into a Ruby object: obj = YAML.load(str).
3. Convert that object to a Hash: h = obj.to_unsafe_h.
4. Write that Hash back to a YAML string: str = h.to_yaml.
5. Put that string back into the database to replace the old one from (1).

Note the to_unsafe_h call in (3). Just calling to_h (or to_hash for that matter) on an ActionController::Parameters instance will give you an exception in Rails5, you have to include a permit call to filter the parameters first:

h = params.to_h                   # Exception!
h = params.permit(:whatever).to_h # Indifferent access hash with one entry

If you use to_unsafe_h (or to_unsafe_hash) then you get the whole thing in a HashWithIndifferentAccess. Of course, if you really want a plain old Hash then you'd say:

h = obj.to_unsafe_h.to_h

to unwrap the indifferent access wrapper as well. This also assumes that you only have ActionController::Parameters in social_media so you might need to include an obj.respond_to?(:to_unsafe_hash) check to see how you unpack your social_media values.

You could do the above data migration through direct database access in a Rails migration. This could be really cumbersome depending on how nice the low level MySQL interface is. Alternatively, you could create a simplified model class in your migration, something sort of like this:

class YourMigration < ...
  class ModelHack < ApplicationRecord
    self.table_name = 'clubs'
    serialize :social_media
  end

  def up
    ModelHack.all.each do |m|
      # Update this to match your real data and what you want `h` to be.
      h = m.social_media.to_unsafe_h.to_h
      m.social_media = h
      m.save!
    end
  end

  def down
    raise ActiveRecord::IrreversibleMigration
  end
end

You'd want to use find_in_batches or in_batches_of instead all if you have a lot of Clubs of course.

---

If your MySQL supports json columns and ActiveRecord works with MySQL's json columns (sorry, PostgreSQL guy here), then this might be a good time to change the column to json and run far away from serialize.

Frazzle

Answer 2

Extending on short's reply - a solution that does not require a database migration:

class Serializer
  def self.load(value)
    obj = YAML.load(value || "{}")
    if obj.respond_to?(:to_unsafe_h)
      obj.to_unsafe_h
    else
      obj
    end
  end
  def self.dump(value)
    value = if value.respond_to?(:to_unsafe_h)
      value.to_unsafe_h
    else
      value
    end
    YAML.dump(value)
  end
end

serialize :social_media, Serializer

Now club.social_media will work whether it was created on Rails 4 or on Rails 5.

Ursas

thanks @jpw - I edited my answer to handle the nil case. – Ursas

Answer 3

The reply by @schor was a life-saver, but I kept getting no implicit conversion of nil into String errors when doing the YAML.load(value).

What worked for me was:

class Foo < ApplicationRecord
  class NewSerializer
    def self.load(value)
      return {} if !value #### THIS NEW LINE
      obj = YAML.load(value)
      if obj.respond_to?(:to_unsafe_h)
        obj.to_unsafe_h
      else
        obj
      end
    end
    def self.dump(value)
      if value.respond_to?(:to_unsafe_h)
        YAML.dump(value.to_unsafe_h)
      else
        YAML.dump(value)
      end
    end
  end

  serialize :some_hash_field, NewSerializer
end

I gotta admin the Rails team totally blindsided me on this one, a most unwelcome breaking change that doesn't even let an app fetch the "old" data.

Parabasis

Answer 4

The official Ruby on Rails documentation has a section about upgrading between Rails versions that explains more about the error you have:

ActionController::Parameters No Longer Inherits from HashWithIndifferentAccess
Calling params in your application will now return an object instead of a hash. If your parameters are already permitted, then you will not need to make any changes. If you are regardless of permitted? you will need to upgrade your application to first permit and then convert to a hash.

params.permit([:proceed_to, :return_to]).to_h

Spiffy

Thanks. I'm still a bit confused though. I'm not calling params but club.social_media['facebook'] which is what causes the error because social_media isn't a hash but and ActionController::Parameter object. – Olivette

Answer 5

Run a migration on Rails 4 to prepare the data for Rails 5.

We're going through the exact same thing, except we serialize as ActiveSupport::HashWithIndifferentAccess instead of just Hash, which I recommend doing, but I'll provide my answer here for just a simple Hash.

If you have not yet upgraded to Rails 5, which I hope you haven't and your tests have uncovered this issue, you can run a migration on the Rails 4 branch that will get your data ready for Rails 5.

It essentially re-serializes all of your records from ActionController::Parameters to Hash while in Rails 4 and ActionController::Parameters still inherits from HashWithIndifferentAccess.

class ConvertSerializedActionControllerParametersToHashInClubs < ActiveRecord::Migration

  disable_ddl_transaction! # This prevents the locking of the table (e.g. in production).

  def up
    clubs = Club.where.not( social_media: nil )

    total_records = clubs.count

    say "Updating #{ total_records } records."

    clubs.each.with_index( 1 ) do |club, index|
      say "Updating #{ index } of #{ total_records }...", true
      club.social_media = club.social_media.to_h
      club.social_media_will_change!
      club.save
    end
  end

  def down
    puts "Cannot be reverse! See backup table."
  end

end

If you have multiple columns that need to be converted, it's easy to modify this migration to convert all of the necessary tables and columns.

Depending on when you do this, your data should be ready for Rails 5.

Grosvenor