How to manage Migrations in a project with multiple branches?

Question

I have an ASP.NET MVC3 project that uses Entity Framework 4.3 with the code-first approach. I use Migrations to keep the database up-to-date.

The project is under source-control and I have a number of branches. What I just realized is that there will be a problem when I want to merge one of my branches into the master. Since I have created migration-files in both branches, there will be overlapping migrations when I merge, which will probably cause conflicts.

Is there a good way to manage Migrations in a project with multiple branches?

Update

One way would be to merge, then delete all migration-files created while the branches were separate, and then create one new migration file that holds all changes from the time the branch was created until it was merged back in. This would work for the dev-environment where you can dump the database and re-build it with all the migration-files. The problem then would be the live-environment. Since you couldn't roll back to the time the branch was created without the risk of loosing data, there will be a conflict when you try to use your new migration-file to update the live database.

Sang

Answer 1

There is a much better solution for handling entity framework migration merge conflicts on a similar question.

All you need to do after a merge is to re-scaffold the meta data of the migration in the target branch. That is you do not rescaffold the up/down code, just the state in the resx-file.

add-migration [the_migration_to_rescaffold_metadata_for]

This almost always works. The procedure will fail if a different migration in the merge have changed the database in such a way that the migration is no longer runnable or gives an unexpected result. That being said - I believe that to be a very rare case as most migrations should be auto-generated, or at least not be dependent on other tables that are not changed in the migration itself as well.

One such case where rescaffold state would fail could be:

• Column foo is an int and rows contain [0, 1, 2]

• Migration A from branch A change foo to boolean (0 will become false automatically and > 0 will become true)

• Migration B from branch B change foo to string. It expects it to be an int but it is a boolean, the migration will succeed though. Data will be lost since when migration B was created the rows would contain ["0", "1", "2"]. When migration A altered column to boolean (and did so successfully and with expected result) the rows will now contain ["0", "1", "1"] instead and Migration B will have a different end result than what was observed in Branch B.

There are probably more edge cases where things could go wrong with the solution. But if migrations up/down code is not dependent on things changed by another migration in the merge it should work well to just update the metadata in the migrations.

Aylsworth

This only works if you are merging from branch B to branch A, then pushing to master; if branch A is already pushed to master and deployed to live environment, and you try this from branch B, it will fail. – Driblet

Please elaborate on "This" in "This only works" and "try this from branch B". If you already have A in master, you could merge master to B, rescaffold, then push B to master. You might have a problematic use-case, elaborate a bit on what you have in mind. – Aylsworth

There is a flag called "IgnoreChanges" that you would probably want to use if you go this route – Latinism

I did different migration changes in branch A and B, then I merged branch B to A. This caused an error about Models not being the same. I did as proposed above to create a new migration and giving it a proper name. Since the scaffolded code contains a field that was already added by previous migration, i emptied both up/down methods, saved, and ran Update-Database. This fixed the issue – Latchstring

"I did as proposed above to create a new migration" - the command should replace the state of last migration. In your case the migration made in branch A works fine, right? The migration in branch B that you merged to branch A does not work, since the state in migration B does not match the state after the merge. They command you should run would be add-migration the-full-name-of-the-migration-that-was-merged-from-branch-b which will replace the base state in the B migration to match the last migration in branch A. You should not make a new migration. – Aylsworth

The accepted answer says “the accepted answer is incorrect”. I feel dizzy ;) – Bougainville

Answer 2

Edit: a colleague of mine discovered an easier with to do this, I left my original answer at the bottom for completeness.

(VERY IMPORTANT) migrations in live environment must not conflict with ones in your current branch, otherwise you need to redo all your migrations and resolve data model change conflicts by hand.

1. restore your development database with live environment data
2. run update-database, it should run migrations from your branch, and complain about 'unable to update database to match the current model blah blah..'
3. run add-migration MergeBranchBToMaster -ignoreChanges, this will create an empty migration.
4. run update-database again
5. push your changes

The magic in step 3 basically tells EF to shutup about mismatched models, hence be very sure that your migrations do not conflict with ones in live environment. If they do, you can always create SQL scripts for pushing the missing migrations (which is actually the preferred method).

Original Answer

I have found a fairly straight-forward solution based on @Ladislav Mrnka's answer. This will work with live environment[1], you just have to be careful not to change any deployed migrations.

1. Before Merge, take note of the migration you added (MyMigration), and its previous migration (BaseMigration)

2. Merge branches in git

3. Open Package Manager Console, and run: UPDATE-DATABASE -TargetMigration:BaseMigration. This will revert your database to the state before any of the conflicted migrations are applied

4. Delete your local migration (MyMigration)

5. Run: UPDATE-DATABASE. This will apply all newer migrations done in other branches.

6. Run: ADD-MIGRATION MyMigration. This will re-generate your local migration based on current state of database, like git -rebase.

7. Run: UPDATE-DATABASE. Update database with you local migration.

This also works if you have multiple local migrations, but it will merge them all into a single one.

[1] by working with live environment, I mean that the generated migration can be applied to live environment which may already have some/all of the other branches' migrations applied. The steps themselves are purely for development purpose.

Driblet

How can you revert a live database? If they've been using this code base with said migration, reverting will leave the application in an inconsistent state and possibly eliminate user data. – Varicocele

those steps are not meant to be used against live environment, I added note to explain what it means. – Driblet

The empty migration is a pretty good solution – Daciadacie

Answer 3

Merging migrations is IMHO manual task. Part of migration code is auto-generated and we usually don't merge auto-generated code - instead we run autogeneration again after the merge.

Until ADO.NET team provides some recommendation I would follow simple principle:

• Before you do the merge revert the master database to the version used prior to branching
• Merge your branches
• Exclude migration classes created after branching from merged assembly
• Add a new migration for merged code base which will migrate your database in the state prior to branching to the state after merging branches
• If your excluded migration classes contain some customization merge them to the new migration class
• Run migration to migrate your database to current merged version

If your branches contained multiple migration steps (version) you will lose them and you will end with two versions - prior to branching and after merging.

Edit:

It will not work in live environment. The problem here would be the development process itself. If you have live environment you should keep its branch untouched (except minor bug fixes). If you continue development in that branch with production deployment and in the same time you build another version in separate branch without continuous integration (= continuous merging changes back to the main branch to integrate your new development with the main code base) you have a big problem. I think migrations in general cannot handle this.

The only option in such case would probably be removing all migrations from merged solution and deleting MigrationHistory table from the database. Than you can enable migrations on the project again and add initial migration to use your current database as starting point = no way back to previous version because no information about previous migrations will exist.

Interlace

Thanks for your answer! Updated my question with a similar thought just as you wrote your answer. Do you have any ideas on how to manage the live-environment? See my updated question for some more info on what I mean. – Sang

Thank you for the clarifications. In my case, since I develop new features (that are not yet ready for production) in a separate branch, I guess the solution would be to continuously merge the master-branch into my separate branch, until the separate branch is ready to be merged back into the master. – Sang

Ouch, this is a big hurt for us. We recently had to push a "hotfix" to the live environment that included a migration to add a new table. The migration in dev is migrating from a different state than the one on live. – Nippur

@Alex Ford You can have the same migration up/down code in two different branches, but two different states for that migration in the resx-file. See my answer. – Aylsworth

I too vote for removal and regenerating of migrations when merging from feature branch to master branch. Feature branch migrations should always be generated with timestamps that follow after all the migrations in the master branch, which might already gone live. Of course, if you merge master into feature regularly (and you should), this means that you should also regenerate your migrations to be timestamped after master branch migrations. – Subantarctic

Answer 4

Rowan Miller has made a great video about this topic on channel 9: Migrations - Team Environments. It refers to entity framework 6.

It describes a scenario where first developer A and B are working on the same model and A checks in first. Now developer B has to deal with the problems he has when he gets the latest version from A.

This is essentially the same like having conflicts between different branches, because the general problem is merging migration changes done it the same time but effectively having a different source state of the model.

The solution is:

• When resolving the conflicts of the version control system, developer B has to accept both changes from himself and developer A.
• An UpdateDatabase command of developer B would still fail at this time (Error message: "Unable to update database to match the current model because there are pending changes...")
• Developer B has to create an "empty migration" using the IgnoreChanges option:

Add-Migration NameOfMigration -IgnoreChanges

Then the UpdateDatabase command will succeed.

Source of the problem

The source of the error occurring when updating the database is because EF stores a snapshot of the model a migration refers to in the resx file within the migration file.

In this case developers B snapshot of the "current model" is not correct after getting / merging the changes made by developer A.

Claqueur

The video explains it all. This should be the accepted answer, in my oppinnion. – Polky

Answer 5

I have put some thought into this and I hope I will contribute to the different opinions and practices presented here.

Consider what your local migrations actually represent. When working locally with a dev database, I use migrations to update the database in the most convenient way possible when adding columns etc to tables, adding new entities etc.

So, Add-Migration checks my current model (let's call it model b) against my previous model (model a) and generates a migration to go from a => b in the database.

To me it makes very little sense to try and merge my migrations with anyone elses migrations, if everyone indeed has their own database and there then exists some kind of stage / test / dev / production database servers in the organization. This all depends on how the team has it set up, but it makes sense to insulate each other from changes that other people make if you want to truly work in a distributed manner.

Well, if you work distributed and have some entity, Person, for example, that you work on. For some reason, lots of other people are also working on it. So, you add and remove properties on Person as needed for your particular story in the sprint (we're all working agile here, aren't we?), like Social Security number that you first made into an integer because you aren't that bright and then to a string etc.

You add FirstName And LastName.

You are then done and you have ten weird up and down migrations (you probably removed some of them while working since they were just crap) and you fetch some changes from the central Git repo. Wow. Your colleague Bob also needed some names, maybe you should've talked to each other?

Anyways, he has added NameFirst and NameLast, I guess... so what do you do? Well, you merge, refactor, change so it has more sane names... like FirstName and LastName, you run your tests and check his code, and then you push to the central.

But what about the migrations? Well, now would be the time to make a migration moving the central repo, or the branch "test" more specifically, contain a nice little migration from its model a => model b. This migration will be one and only one migration, not ten weird ones.

Do you see what I'm getting at? We are working with nice little pocos and the comparisons of them constitute the actual migrations. So, we shouldn't merge migrations at all, in my opinion, we should have migrations-per-branch or something like that.

In fact, do we even need to create the migration in the branch after merge? Yes, if this database is updated automatically, we need to.

Gotta work some more, those are my thoughts on this, at least.

Freethinker

That is an interesting thought indeed. So I guess what you are saying is that the migration files don't belong under source control at all? – Sang

One use-case is where the migration contains logic of some kind. If you merge your modifoed pocos to different branches, each of those targets would have to have similar migrations created. What happens if you forget that non-automatically generated part of the migration? I agree though that most migrations are automatically created and could be created easily in target branch when needed. – Aylsworth

Answer 6

Consider using a different migration library that doesn't cause these conflicts, such as FluentMigrator or Migrator.NET.

I don't think EF migrations are really ready for general use with branches & merges - it's a lot of work, and too easy to make nasty mistakes.

Cleft

Answer 7

I think what @LavaEater is saying makes a lot of sense. I'm implementing a branching strategy (Development, Main, Release) and aligning it with the environments in the development, QA and release process.

• Development branch - Local development
• Main branch - Merge changes from Development branch and deploy to my Staging environment (an Azure website and SQL database)
• Release branch - Merge changes from Main and deploy to Production environment (another Azure website and SQL database)

I've come up against the problem discussed above and in my opinion the complications around migrations and the potential workarounds introduce a great deal of risk into the release process. Executing independent migrations in Development, Main and Release effectively means that the schema I included in the build in Dev is not the schema that goes into QA on Staging and the schema that QA signs off on Staging is not the schema that is deployed to Live (unless I follow one of the suggested solutions which I'm sure would work but may be error prone).

To echo @LavaEater - what is the real benefit I get from EF code first? Personally, I think it's the ease with which I can generate a schema from code (and potentially tweak the automatically generated migrations if I want to). After that, migrations are a complication of what should be a simple deployment process.

My current thinking is to use code first to generate the migrations in development and then either:-

• Option A) - Use Update-Database -script to script up the schema changes and put them under source control. There is still some potential for conflicts if 2 people are amending the same model but I think that it's easier to manage.

• Option B) - Use something like SQL Compare to generate schema change scripts. This is potentially more flexible and transparent as I like to see exactly what schema changes I'm applying to my Production database (call me paranoid).

Am I missing something? I imagine there will be some configuration to do to disable code first migrations in the Main and Release branches (on the assumption that the DB will be created and updated by scripts). Other than that it feels like a safe solution but I would value a 2nd opinion.

Schroer

I agree, and the missing piece is: DevOps should track current-migration and compare your schema/resx against what it has. If it find a "migration schema conflict" (not a code conflict!) it should notify the devs in the pull request. – Trude

Answer 8

I know this is an old post but I just had a similar problem - more then one commits to local branch, for some reason I deleted a migration at commit 'x' containing table creation and didn't want to recreate that manually. I ended up deleting the ...ContextModelSnapshot.cs replacing it by the file from origin branch (e.g. develop) and executing add-migration to generate a clean migration from the latest model

Redware

Answer 9

My solution is before doing the migration in two different branches: We have two different branches, A and B. We will also have C branch, we will call it 'migration branch'.

a table will be added in branch A and a column will be added in branch B. We will make those migrations in the C branch and brach A and B will merge this branch to themselves

In this way, we will prevent migration conflicts in both branches.

Election