MongoMapper and migrations

Question

I'm building a Rails application using MongoDB as the back-end and MongoMapper as the ORM tool. Suppose in version 1, I define the following model:

class SomeModel
  include MongoMapper::Document
  key :some_key, String
end

Later in version 2, I realize that I need a new required key on the model. So, in version 2, SomeModel now looks like this:

class SomeModel
  include MongoMapper::Document
  key :some_key, String
  key :some_new_key, String, :required => true
end

How do I migrate all my existing data to include some_new_key? Assume that I know how to set a reasonable default value for all the existing documents. Taking this a step further, suppose that in version 3, I realize that I really don't need some_key at all. So, now the model looks like this

class SomeModel
  include MongoMapper::Document
  key :some_new_key, String, :required => true
end

But all the existing records in my database have values set for some_key, and it's just wasting space at this point. How do I reclaim that space?

With ActiveRecord, I would have just created migrations to add the initial values of some_new_key (in the version1 -> version2 migration) and to delete the values for some_key (in the version2 -> version3 migration).

What's the appropriate way to do this with MongoDB/MongoMapper? It seems to me that some method of tracking which migrations have been run is still necessary. Does such a thing exist?

EDITED: I think people are missing the point of my question. There are times where you want to be able to run a script on a database to change or restructure the data in it. I gave two examples above, one where a new required key was added and one where a key can be removed and space can be reclaimed. How do you manage running these scripts? ActiveRecord migrations give you an easy way to run these scripts and to determine what scripts have already been run and what scripts have not been run. I can obviously write a Mongo script that does any update on the database, but what I'm looking for is a framework like migrations that lets me track which upgrade scripts have already been run.

Selfinduced

I think Mongo(/Mapper) might just be too young for that sort of thing. :/ – Incongruous

Migration in terms of schema actually is not a proper concept in Mongo DB since Mongo DB actually doesn't have any schema. You need to just write the data migration script by yourself. – Eu

Answer 1

Check out Mongrations... I just finished reading about it and it looks like what you're after.

http://terrbear.org/?p=249

http://github.com/terrbear/mongrations

Cheers! Kapslok

Oh

For Rails 3, check out this fork of terrbear's mongrations: github.com/TheHiveProjects/mongrations. (As of this writing, that is the fork with the most recent commits.) I had to specify gem 'mongrations', :git => 'git://github.com/TheHiveProjects/mongrations.git' to get it to work. – Smallage

Answer 2

One option is to use the update operation to update all of your data at once. Multi update is new in the development releases so you'll need to use one of those.

Firecure

Answer 3

You can try this contraption I just made, but it only works with mongoid and rails 3 (beta 3) at the moment. http://github.com/adacosta/mongoid_rails_migrations . It'll be upgraded to rails 3 when it goes final.

Abdul

Answer 4

Also another gem for MongoMapper migrations https://github.com/alexeypetrushin/mongo_mapper_ext

Colocynth

Answer 5

Mongrations is a super old gem, completely deprecated. I recommend NOT using it.

Exodus is a really cool migration framework for Mongo, that might be what you want:

https://github.com/ThomasAlxDmy/Exodus

Bruckner

Answer 6

We just build this one: https://github.com/eberhara/mongration - it is a regular node module (you can find it on npm).

We needed a good mongodb migration framework, but could not find any - so we built one.

It has lot's of better features than the regular migration frameworks:

• Checksum (issues an error when a previosuly ran migration does not match its old version)
• Persists migration state to mongo (there is no regular state file)
• Full support to replica sets
• Automatic handle rollbacks (developers must specify the rollback procedures)
• Ability to run multiple migrations (sync or async) at the same time
• Ability to run migrations against different databases at the same time

Hope it helps!

Brazil

Answer 7

Clint,

You can write code to do updates -- though it seems that for updating a record based on its own fields is not supported.

In such a case, I did the following and ran it against the server:

------------------------------
records = Patient.all()

records.each do |p|
  encounters = p.encounters
  if encounters.nil? || encounters.empty?
    mra = p.updated_at
    #puts "\tpatient...#{mra}"
  else
    mra = encounters.last.created_at
    #puts "\tencounter...#{mra}"
  end
  old = p.most_recent_activity
  p.most_recent_activity = mra
  p.save!
  puts "#{p.last_name} mra: #{old} now: #{mra}"
end
------------------------------

Marigolde

Answer 8

I bet you could hook into Activerecord::Miration to automate and track your "migration" scripts.

Wally

Answer 9

MongoDB is a schema-less database. That's why there are no migrations. In the database itself, it doesn't matter whether the objects have the key :some_key or the key :some_other_key at any time.

MongoMapper tries to enforce some restrictions on this, but since the database is so flexible, you will have to maintain those restrictions yourself. If you need a key on every object, make sure you run a script to update those keys on pre-existing objects, or handle the case of an object that doesn't have that key as you come across them.

I am fairly new to MongoDB myself, but as far as I can see, due to the flexibility of the schema-less db this is how you will need to handle it.

Macri

I take issue with this argument: "MongoDB is a schema-less database. That's why there are no migrations." Though MongoDB does not enforce a schema across your documents, in practice you will likely want your application to enforce a schema, to some degree. If you define "migration" a little more broadly it is easy to see how a MongoDB-backed Web application might need migrations. It is true that one kind of migration is a "formal" schema migration. But there are other kinds of migrations that are still very important: adding keys, renaming keys, changing data, etc. – Bordy