Deploying a PHP webapp to multiple EC2 instances behind a Elastic Load Balancer

Question

my question is basically two questions, but since they are closely related I thought that it makes sense to ask them en-bloque.

Case:
I am running a webapplication, which is distributed over multiple AWS EC2 instances behind a AWS Elastic Load Balancer

Intended goals:
a) When deploying new app code (php) it should be automatically distributed to all EC2 instances.
b) When new EC2 instances are added, they should automatically "bootstrap" with the latest appcode

My thoughts so far:
ad a)
phing (http://phing.info) is probably the answer for this part. i would probably add multiple targets for each EC2 instance and when running a deploy it would deploy to all machines. probably unfortunately not in parallel. but that might be even beneficial when scripting it in a way where the EC2 instance is "paused" in the load balancer, upgraded, "unpaused" again and on to the next instance.

ad b)
not sure how i would achieve that. in a conventional "hardware based setup" i probably had a "app code" volume on a network storage device and when adding a new server i'd simply attach that volume. when deploying new appcode i had just one deploy operation to this volume. so i need some "central storage" from where the newly bootstrapped machine/instance downloads it's appcode. i thought about git, but after all git is not a deploy tool and probably shouldn't be forced to be used as one.

I'd be happy to see your setups for such tasks and hear your hints and ideas for such a situation.

Thanks,

Joshua

Stegall

Answer 1

This could be made using phing. However, I'm not sure why you want new instances to automatically retrieve the appcode? Do you get extra instance very often? And in order for a) to deploy code to several instances, it would still need to know them?

This setup requires a master deploy server and uses a push strategy. The master server needs phing, any required phing packages, and optionally ssh keys for the EC2 instances.

Suggestion for a)

(This is just a general outline of the phing tasks required)

• Retrieve instance list (either config file or suuplied parameters)
• Export appcode from repository to master server(e.g. SubVersion)
• Tar appcode
• scp tarball to all EC2 instances (to a deploy folder)
• With rsh unpack tarball on EC2 instances
• With rsh update symbolic link on EC2 instances so webserver folder points at new deploy folder
• Clear any caches on webserver

The above could be called after you have made a new release.

Suggestion for b) This could be achieved by running the phing script every couple of hours, have it login to the EC2 instances and check for the appcode. If it doesn't find it, it will deploy the newest final release. This of course, requires the EC2 instances to be setup correctly in regard to webservers, config files, etc. (However, this could also be achieved by remote shell via phing).

I've previously used similar setups, but haven't tried it with services like EC2.

Gynophore

Answer 2

a) Take a look a Capistrano and since you're not using Ruby (and RoR) use the railsless-deploy plugin. Capistrano can deploy to multiple servers.

b) Haven't gotten any experience with this, but I wouldn't be surprised if there isn't a plugin/gem for Capistrano that can pretty much do this for you.

Hus

Thanks for your reply! I have already checked out Capistrano. What bothers me a bit is, that it is Ruby-based and I don't have any previous experience with Ruby. Hopefully there will be some more replies, maybe even from people who have already solved such tasks. This might be a very interesting topic for many people running scalable (or hopefully scaleable) webapps. – Stegall

I understand where you're coming from :-). I once asked a similar question for a Lua or Perl based deployment tool as I didn't knew any Ruby and was too lazy to learn it. Eventually I gave Capistrano a shot and am still using it. It may take a bit of time to set it up but it simplifies deployment a lot. – Hus

Answer 3

I believe phing is a good tool for this. Capistrano might also help.

I deploy my application in a very similar environment. So far, I've been using simple bash scirpts, but I'll probably be moving towards a phing based solution, mostly because of the difficulty involved in developing shell scripts (you have to know a new syntax that's not very flexible, not to mention cross-platform) and the availability of parallel processing, which is present in phing.

Joby

Answer 4

Phing is a great tool for managing your deployment tasks, that pretty much covers most of (a).

We use OpsCode Chef for the infrastructure management. It has a deploy_revision target that supports both SVN and Git repos for application code deployment as well, and knife (the main Chef command-line tool) has an EC2 plug-in that allows you, with one command, to spin up a new EC2 instance with whatever roles and environments you have defined in Chef, and deploy your application code.

For managing this with multiple EC2 instances behind an ELB, we have used the Python boto library to write very simple scripts that connect to the ELB, get a list of all instances attached to that ELB and one-by-one removes an instance from the ELB, runs the Chef update on that instance (which deploys the new application code and any machine configuration changes), re-attaches the instance to the ELB and moves on to the next instance.

Suttee