Heroku: web dyno vs. worker dyno? How many/what ratio do I need?

Question

I was curious as to what the difference between web and worker dynos is on Heroku. They give a one sentence explanation on their pricing page, but this just left me confused. How do I know how many to pick of each? Is there a ratio I should aim for? I'm pretty new to this stuff, so can someone give an in depth explanation, or maybe some sort of way I can calculate how many and which kind of dynos I would need?

Also, I'm confused about what they mean by the amount of hours for each dyno.

http://www.heroku.com/pricing

I also happened upon this article. As one of their suggested solutions, they said to increase the amount of dynos. Which type of dyno are they referring to here?

http://devcenter.heroku.com/articles/backlog-too-deep

Unspent

Answer 1

Your best indication if you need more dynos (aka processes on Cedar) is your heroku logs. Make sure you upgrade to expanded logging (it's free) so that you can tail your log.

You are looking for the heroku.router entries and the value you are most interested is the queue value - if this is constantly more than 0 then it's a good sign you need to add more dynos. Essentially this means than there are more requests coming in than your process can handle so they are being queued. If they are queued too long without returning any data they will be timed out.

There's no ideal ratio I'm afraid, you could have an app doing 100 requests a second needing many web processes but just doesn't make use of workers. You only need worker processes if you are doing processing in the background like sending emails etc etc.

ps Backlog too deep would be a Dyno web process that would cause it.

UPDATE: On March 26 2013 Heroku removed queue and wait fields from the log out put.

queue and wait fields have been removed from router log messages. Also, the Heroku router no longer sets X-Heroku-Dynos-In-Use, X-Heroku-Queue-Depth and X-Heroku-Queue-Wait-Time HTTP headers for incoming requests.

Kink

To look at heroku router logs, do heroku logs -p router --tail – Cruel

I dont see a queue val I see dyno=web.1 connect=2ms service=4ms status=200 bytes=43 – Regin

@nathan that's a great tip, -p worker lets me monitor workers oonly, didnt see that in heroku docs – Acton

Another update: might be worthwhile for those on a budget to use Hirefire, which autoscales your workers (or any other autoscale solution). Keep in mind that these usually have pitfalls though -- Hirefire at least USED to have a problem with running scheduled jobs instead of just plain background jobs. – Unspent

Why did they remove them? – Empty

You can still get this information by enabling the Heroku Labs add-on log-runtime-metrics. Run the following command to do so, heroku labs:enable log-runtime-metrics. Read more here: devcenter.heroku.com/articles/log-runtime-metrics – Scottiescottish

https://mcmap.net/q/243837/-thin-vs-unicorn-on-heroku - Heroku has gone to random routing, so some dynos can have queues stacking up while other dynos are free. Avoid this by making sure that all requests are handled very quickly in your web dynos. – Opportunity

Answer 2

Dynos are basically processes that run on your instance. With the new Cedar stack, they can be set up to execute any arbitrary shell command. For web applications, you generally have one process called "web" which is responsible for responding to HTTP requests from users. All other processes are what were previously called "workers." These run continuously in the background for things like cron, processing queues, and any heavy computation that you don't want to tie up your web processes. You can also scale each type of process, so that multiple processes of each type will be booted up for additional concurrency. The amount of each that you use really depends on the needs of your application and the load it receives. You can use tools like the New Relic plugin to monitor these things. Take a look at the articles on the Process Model and the Procfile in Heroku's dev center for more details.

Progressist

"Dynos are basically processes that run on your instance." This is an incorrect statement. Dyno exist on different instances. – Purifoy

Answer 3

A number of people have mentioned that there is no known ratio and that the ratio of web workers to 'background' workers you will want is dependent on how you designed your application - that is correct. However I thought it might be useful to add that as a general rule of thumb, you want your web workers - and thus the controller actions they are serving - to be lightning quick and very lightweight, to reduce latency in response times from browser actions. If there is some browser action that would require more than, say, about a half a second of real time to serve, then you will probably want to architect some sort of system that pushes the bulk of that action on to a queue.

You would then design an offline worker dyno(s) that will service this queue. They can take much longer because there are no HTTP responses pending on their output. Perhaps the page you rendered from the initial browser request that pushed the action will serve some Javascript that starts a thread that checks to see if the request has finished every 5 seconds, or something along those lines.

I still can't give you a ratio to work with for the same reason others have given, but hopefully this helps you decide how to architect your app. (I should also mention this is just one design out of many valid ones.)

Gaylene

Answer 4

https://mcmap.net/q/243837/-thin-vs-unicorn-on-heroku - Heroku has gone to random routing, so some dynos can have queues stacking up (while they serve a lengthy request) while other dynos are free. Avoid this by making sure that all requests are handled very quickly in your web dynos. This will reduce the number of web dynos you need, while requiring more worker dynos.

You also need to care about your web app supporting concurrency, which only some Rails configs do - try Unicorn, or carefully-written code (for I/O that doesn't block the EventMachine) with Thin.

You probably have to try, rather than calculate, to see how many dynos of each kind you need. Make sure their New Relic reports the dyno queue - see the above link.

Opportunity

Answer 5

Short answer is that you need as many as you need to keep your queues down.

As John describes, if you start seeing a queue in your logs then you need more dynos. If you start seeing your background queues getting too long (how you get this info is dependant on what you have implemented) then you need more workers.

There is no ratio as it's very much dependent on your application design and usage.

Purifoy

Ok, thanks. I'm assuming by dynos you mean web dynos. Also, how do I check for a queue in my log? More specifically what I'm asking for is how do I identify if things are piling up when I read my log? I'm a Rails developer so I often deal with running a local server and reading those logs, but I'm not sure I would know how to identify a queue if I saw one. – Unspent

my answer describes how to identify queue size - tail you logs on heroku and look for router entries and the queue= value. Your local logs won't help you - you need to be using heroku logs -f from command line. – Kink

@JohnBeynon Ok, thanks. Didn't realize this until later rereading. – Unspent