UPDATE: It's now been just over three years since I first posted this question and answer. I still prefer AWS for all new deployments of a professional or serious horsepower nature (that is, if I don't self-host), but I also regularly deploy demo and tutorial apps to Heroku. I haven't tried any of the many VPS providers that have popped up such as Linode or DigitalOcean, but generally hear good things about them.
The key thing that keeps me from choosing Heroku for all my apps is cost. Since most of my indie projects outside of work are things where I'd prefer to absolutely minimize costs, AWS remains the better deal between AWS vs. Heroku. AWS (or any VPS provider for that matter) has the nice side effect of teaching you the OS along the way, which turns out to be hugely valuable in the long term.
=======================
So, two years later, here's my update. I've used three services for hosting, and here's my take on each of them (I actually love them all, but for various reasons).
This was the first VPS host I tried, and I loved them. The people there were amazing, support was awesome, and it had a really cool grassroots kind of feel. Now that VPS as a solution is more mainstream, and Rackspace has long since purchased Slicehost, I feel that the service offering is still awesome. If you want a simple way to setup a server, plenty of Linux distro choices, and control over your server, this is an awesome option.
Love these guys too. I built a hobby app that is hosted there, rpglogger.com (which as of Nov. 2012 has actually migrated to Amazon Web Services), and developing and deploying to Heroku is a no-brainer. I really like working on Heroku for two reasons:
- It's dead simple to setup. It really is as easy as they say, in my experience, to get an app running on their platform.
- A single dyno (web server instance) is free. So hobby apps, and smaller apps basically get free hosting. It's not just for hobbies though - their plugin architecture is second-to-none, making the addition of 3rd party plugins such as NewRelic, Exceptional, and anything else on their platform a matter of just a few clicks.
You absolutely cannot beat Heroku for ease of use. Deploying an updated version of your app is literally as simple as pushing to your git repo. Heroku isn't necessarily cheap (for anything other than the small app), but if you're in a situation where you believe developer time is more valuable than having control over the server, then this is an amazing option. You can always migrate your app to any other platform anyhow, if it gets big, or the needs of the app vs. the cost of Heroku no longer make sense for you.
I do quite a few small apps, and AWS reserved instances are awesome. For $60 I can basically get a reserved instance for an entire year. That one server is enough to run 3-4 small apps on the same machine, with more optimized memory usage, and the ability to run multiple web server instances (vs. Heroku's one free dyno, though I hear you can custom config your Heroku dyno using unicorn to get more scalability). Basically, AWS scales really well, and lets you share a server among multiple small apps, or spread a larger app across multiple servers.
On top of that initial cost for the reserved instance, I only have to pay for bandwidth and other AWS usage (S3, for example). I think AWS is an amazing mix of ultimate scalability, great costs, ultimate control, and for enterprise customers who want to build their entire infrastructure in the sky, it can't be beat. Rackspace Cloud Hosting provides similar services, and they're probably comparable for most things. But if what you want is the Swiss army knife of cloud services, I think AWS is still way ahead of everyone else.
===============
So, that being said, I started on Slicehost, then went to AWS, then tried Heroku, and today I spend most of my time back on AWS.
AWS is the kind of platform that, after you invest a little time in setting up your collection of VPS machines, it often makes sense to stay on this platform and leverage their ever increasing set of tools.
Granted, it took me two years of trying several options, and trying every level of management from fully managed servers (i.e. Heroku, where you don't even think of the server, just the app) to fully controlled servers (Slicehost and AWS). After all that I've come to this point where I'm ready to manage my own machines in order to get the flexibility and low costs that I want.
Through automation, the actual management of the servers on AWS becomes a non-event, so I don't spend my time constantly patching my machines, or doing other sysadmin tasks. I just check periodically to see if my servers need reboots, I set them to automatically install all security updates (I happen to deploy to Ubuntu servers), which means I spend 99% of my time (at least day-to-day) writing the application - not managing the servers (managing services is instead an occasional task of a few day's work, and then nothing else for months) - which is where I want to spend my time as a developer.