Symfony 4: how to safely 'join' migrations together? / remove migrations in between the first and last?

Question

I was recently tweaking around with my DB and noticed that I made 5 migrations on top of my existing 2. So 7 in total, m1, m2, ..., m7. From these 5, I only want the latest version. So I was thinking I could remove m3, m4, m5, m6, m7 and migrate again, which will result in m3 that already contains the final version that I'm happy with. Now I have never done this before and was wondering what is the safest way to do it?

I'm using PHPStorm. Could I manually remove them (right click in PHPStorm then Delete) and then go to the command line and do php bin/console doctrine:migrations:migrate? Or do I need to do php bin/console doctrine:schema:update? Or is this really a bad idea?

Forsook

Answer 1

No, generally you shouldn't do either of those. Let me start by clearing up what seems like a misconception: a migration doesn't contain a full database schema, only the queries necessary to bring the database from one version to the next. This system allows for database changes to be managed and versioned. So you should be starting to see some problems with this:

• A migration might depend on a previous one. E.g. v1 adds an entity and v2 adds a new field. If you remove v1, v2 will be broken and needs to be removed as well.
• Since a database migration usually goes hand in hand with an object model change, you will have to revert the model as well, otherwise you'll find mapping problems with non existent fields.

This might be acceptable during the design phase when you don't have any functionality yet. You can revert an individual migration using bin/console doctrine:migrations:execute --down <version>. This is usually done while testing changes if adjustments need to be made. But usually when that change hasn't been committed yet.

Doctrine keeps track of migrations using a database table called migration_versions. By naming them with a date, it can order them and apply them sequentially. Whenever a migration is executed, it adds the migration name to this table. When you roll it back, it deletes it from the table, along with the fields in the migration itself. Keep in mind that even though you can rollback a migration, it doesn't mean everything will be as it was. If a migration drops a column, the column will be recreated on rollback, but the data will be lost.

As for "can it be done"? Yes. If you really really want to, have read the documentation thoroughly and are aware of all this.

So, since your question is about merging migrations, let's tackle your actual options:

• Could I manually remove them (right click in PHPStorm then Delete) and then go to the command line and do php bin/console doctrine:migrations:migrate?

No, this won't work. migrate will apply available migrations. They won't be any and, as explained, the revisions will still be in the table and its changes applied.

• Or do I need to do php bin/console doctrine:schema:update?

This won't do anything either, since it will compare the model with the database and find that they match.

In any case, you will need to revert them first and then create an equivalent one. The command for that is doctrine:migrations:diff. This will compare the model with the schema and generate a migration to get the database in sync. And for this to work you will need to execute --down your migrations first, otherwise they won't be any changes, but potentially losing some data in the process.

If you work in a team, they will be seeing migrations disappear. Some might even be behind in the history and not applied all the migrations. This will soon become a management pain. There is a rollup command that (with my understanding and never actually used it) cleans stale migrations from the table, dumps a full schema and applies it. This will be your best bet, but be aware that this most like will delete your data.

You could also combine your migrations manually. They are just classes with an up and down methods. Combine all function bodies, apply the cleanup process and call it a day.

Now, if you want to do this it shouldn't be much of a problem. Just replace your outdated versions with your new one and warn everybody about it. But if what you want is to do as they never existed at all and keep a neat commit history, then that's when your teammates will potentially want to kill you, as it will involve rewriting history. When you do that, they will have to rebase all their work.

If you want to do it:

• Make backups
• Introduce your changes as early as possible to avoid breaks. Is ok if there are some unused fields in the database for a while, until the model commits catch up. If you do it late and some object need it, someone might be forced to create a migration and break yours in the process.
• Get it right (preferably on the first try). Don't push until you have tested extensively.
• Make backups

Reference for the console commands

Hayley

If you merge it by adding up & down instructions in right order with a comment to note the original migration file, it could be good. It could be a good practice to do before each delivery (from previous delivery only). – Acceleration

Answer 2

There isn't a "safe" way of doing it, but if you haven't deployed the migrations then you can safely throw away a series of auto-generated migrations and regenerate them. Just be careful you don't throw away manual migration SQL at the same time.

Normally during the design stage we will:

1. Use doctrine:schema:update --force until things are fairly stable
2. After they are stable, we reset the development database from a snapshot
3. Run doctrine:migrations:diff and then add manually migration queries where necessary

Output

Will the queries work correctly if, let's say, in migration m3 I say create user table, and then drop it in migration m5 or will it crash? – Forsook

Think of the sorted migrations as a long list of SQL statements which will be run sequentially. There isn't any magic. The main benefit is that as time goes on you can run only the required lines. The second benefit is that you can deploy the required SQL changes along side changes your business objects. If you can run two SQL statements consecutively outside a transaction, then you can run them in two migrations. – Output

Answer 3

May be you need to ignore all your migrations and focus on your last state of database schema. If this is what you are wondering to have in final, then you could just hit: php bin/console doctrine:schema:update --dump-sql to get the corresponding SQL statement, or hit php bin/console doctrine:schema:update --force to apply it actually on database.

Polyzoan

Answer 4

In general yes you can. But you should know about that the Doctrine/Migration save executed migrations in DB . So you should remove unnecessary migration files (possible combine all queries into one), after that update database.

Table name by default "doctrine_migration_version" just delete unnecessary version rows

Clavus

-1 - If you remove migrations from somewhere in the middle, clone the project and then run migrations you're going to have a bad time. Do not do this. Read the accepted answer instead of this one – Luminesce

It is all ok.But question as I realized was "can I remove some migrations in middle if in last migration I have all needed queries." So answer is yes. – Clavus

Still no. For the sake of argument, please setup a random project, create 3 migrations, each creating a table (fruit_* prefix, tables "apples", "pears", "bananas", in order). Execute migrations. Create some index-display logic to show these. Remove "pears" migration. Commit to repo. - Now, clone project under new name (or other computer, whatever). Run migrations. No "fruit_pears" table (and errors if your app logic depended/expected it to exist)... So,unless you want to face the music, the answer is just "no". – Luminesce

Ok you win. But key word was "I have all needed queries". – Clavus