Hot deploy on Heroku with no downtime

Question

A bad side of pushing to Heroku is that I must push the code (and the server restarts automatically) before running my db migrations.

This can obviously cause some 500 errors on users navigating the website having the new code without the new tables/attributes: the solution proposed by Heroku is to use the maintenance mode, but I want a way with no downside letting my webapp running everytime!

Is there a way? For example with Capistrano:

• I prepare the code to deploy in a new dir
• I run (backward) migrations and the old code continue to work perfectly
• I swith mongrel instance to the new dir and restart the server

...and I have no downtime!

Veliz

Answer 1

You could setup a second Heroku app which points to the same DB as your primary production app and use the secondary app to run your DB migrations without interrupting production (assuming the migrations don't break the previous version of your app).

Let's call the Heroku apps PRODUCTION and STAGING.

Your deploy sequence would become something like:

1. Deploy new code to STAGING
   git push heroku staging
2. Run database migrations on STAGING (to update PROD db)
   heroku run -a staging-app rake db:migrate
3. Deploy new code to PRODUCTION
   git push heroku production

The staging app won't cost you anything since you won't need to exceed Heroku's free tier and it would be pretty trivial to setup a rake deploy script to do this for you automatically.

Good luck!

Subcritical

I believe Heroku will keep your old dynos up until compilation of the new slug is complete so your site should stay available while step 3 is running. I suppose there could be some momentary downtime while Heroku cuts the routing over from the old dynos to the new ones but it should be pretty minimal. – Subcritical

I'm not sure its a really good idea to have your staging environment linked to your production db if you're really using it as a staging environment. – Xiphisternum

STAGING is just an example name, I'm not suggesting you should reuse the same environment you use to test pre-production code for this purpose. That would be a bad idea. In this scenario you're creating a dedicated second copy of your production app configuration, specifically to use in the deploy process. You can call it anything. – Subcritical

Answer 2

If you're able to live with two versions of the same app live at the same time, Heroku now has a preboot feature.

https://devcenter.heroku.com/articles/preboot

Seidel

Answer 3

The only method to improve the process somewhat is what this guy suggests. This is still not a hot deploy scenario though:

http://casperfabricius.com/site/2009/09/20/manage-and-rollback-heroku-deployments-capistrano-style/

One thing I would suggest is pushing only your migrations up to Heroku first and running them before you push your codebase. This would entail committing the migrations as standalone commits and manually pushing them each time (which is not ideal). I'm very surprised there is not a better solution to this issue with all of the large apps hosted on Heroku now.

Gripper

Answer 4

You actually will have some downtime when Heroku restarts your app. They have a new feature called Preboot that starts up new dynos before taking out the old ones: https://devcenter.heroku.com/articles/labs-preboot/

As for database migrations, that article links to this one on how to deal with that issue: http://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/

Shortcut

Answer 5

I first commit the migrations, run them, then push the rest of the code. Add a single file like so:

git commit -m 'added migration' -- db/migrate/2012-your-migration.rb

Swanson

Answer 6

Heroku can't deploy by capistrano. You are block by tool released by heroku.

The no downtime system is impossible in all cases. How change your schema with big change without stop your server. If you don't stop it, you can avoid some change and your database can be inconsistent. SO the using of maintenance page is a normal solution.

If you want a little solution to avoid problem is a balancing in two server. One with only read database during your migration. You can switch to this instance during your migration avoiding the maintenance page. After your migration you come back to your master.

Esperanzaespial

Hi shingara, I'm sorry but I don't agree with you. I don't want to use load balancing for this: one of the great features of Heroku is the "trasparent" cloud power by necessity and I want to use this feature...To load balance in Heroku I have to mantain two different apps and a read only DB can cause problems to my users. And a no downtime system is not ever impossible. I'm used right the system explained with no downtime. In case of a big change without the possibility of a retro-compatible db schema I can use a maintenance page: but this is the 5% of all my cases... – Veliz

You can avoid the problem described in this answer by using CouchDB, for example. – Safe

Answer 7

Right now I don't see any possibility to do this without downtime. I hate it too.

This console command does it in the smallest amount of time I can think of

git push heroku master && 
heroku maintenance:on && 
sleep 5 && 
heroku run rails db:migrate && 
sleep 3 && 
heroku ps:restart && 
heroku maintenance:off

1. git push heroku master to push the master branch to heroku
2. heroku maintenance:on to put on maintenance so no 500s
3. sleep 5 to let the dynos start up the new code (without it, the migration might fail)
4. heroku run rails db:migrate to do the actual migration
5. heroku ps:restart out of experience the restart makes sure the new dynos have the latest schema
6. heroku maintenance:off turns of the maintenance

You might have to add -a <app name> behind all heroku commands if you have multiple apps.

Just one command will run these in series in terminal on Mac OSX.

Ostmark