Run multiple WorkerRoles per instance

Question

I have several WorkerRole that only do job for a short time, and it would be a waste of money to put them in a single instance each. We could merge them in a single one, but it'd be a mess and in the far future they are supposed to work independently when the load increases.

Is there a way to create a "multi role" WorkerRole in the same way you can create a "multi site" WebRole?

In negative case, I think I can create a "master worker role", that is able to load the assemblies from a given folder, look for RoleEntryPoint derivated classes with reflection, create instances and invoke the .Run() or .OnStart() method. This "master worker role" will also rethrown unexpected exceptions, and call .OnStop() in all sub RoleEntryPoints when .OnStop() is called in the master one. Would it work? What should I be aware of?

Ellingston

Ruben you always edit my code/post ! haha – Ellingston

Answer 1

As mentioned by others, this is a very common technique for maximizing utilization of your instances. There may examples and "frameworks" that abstract the worker infrastructure and the actual work you want to be done, including one in this (our) sample: http://msdn.microsoft.com/en-us/library/ff966483.aspx (scroll down to "inside the implementation")

Te most common ways of triggering work are:

1. Time scheduled workers (like "cron" jobs)
2. Message baseds workers (work triggered by the presence of a message).

The code sample mentioned above implements further abstractions for #2 and is easily extensible for #1.

Bear in mind though that all interactions with queues are based on polling. The worker will not wake up with a new message on the queue. You need to actively query the queue for new messages. Querying too often will make Microsoft happy, but probably not you :-). Each query counts as a transaction that is billed (10K of those = $0.01). A good practice is to poll the queue for messages with some kind of delayed back-off. Also, get messages in batches.

Finally, taking this to an extreme, you can also combine web roles and worker roles in a single instance. See here for an example: http://blog.smarx.com/posts/web-page-image-capture-in-windows-azure

Patience

Answer 2

Multiple worker roles provide a very clean implementation. However, the cost footprint for idle role instances is going to be much higher than a single worker role.

Role-combining is a common pattern I've seen, working with ISV's on their Windows Azure deployments. You can have a background thread that wakes up every so often and runs a process. Another common implementation technique is to use an Azure Queue to send a message representing a process to execute. You can have multiple queues if you want, or a single command queue. In any case, you would have a queue listener running in a background thread, which would run in each instance. The first one to get the message processes it. You could take it further, and have a timed process pushing those messages onto the queue (maybe every 24 hours, or every hour).

Aside from CPU and memory limits, just remember that a single role can only have a maximum of 5 endpoints (less if you're using Remote Desktop).

EDIT: As of September 2011, role configuration has become much more flexible, now that you have 25 Input endpoints (accessible from the outside world) and 25 Internal endpoints (used for communication between roles) across an entire deployment. The MSDN article is here

I recently blogged about overloading a Web Role, which is somewhat related.

Hookworm

Answer 3

While there's no real issue with the solutions that have been pointed out for finding ways to do multiple worker components within a single Worker Role, I just want you to keep in mind the entire point of having distinct Worker Roles defined in the first place is isolation in the face of faults. If you just shove everything into a single Worker Role instance, just one of those worker components behaving badly has the ability to take down every other worker component in that role. Now all of a sudden you're writing a lot of infrastructure to provide isolation and fault tolerance across components which is pretty much what Azure is there to provide for you.

Again, I'm not saying it's an absolute to strickly do one thing. There's a place where multiple components under a single Worker Role makes sense (especially monaterily). Simply saying that you should keep in mind why it's designed this way in the first place and factor that in appropriately as you plan your architecture.

Entitle

True, but you could have multiple instances of worker roles with many tasks en each for fault tolerance. Windows Azure SLAs are only applicable if you have at least 2 instances of any given role. Money is a great motivator :-) – Patience

Exactly, it's as Eugenio just pointed out. The idea is run WorkerRoles A, B and C in a single master WorkerRole, and put at least two instances of it. Then we get the benefits of Azure as well :P – Ellingston

You still have to provide scheduling and fault tolerance in the master role to ensure, for example, that if inner role A has a bug that is triggered by a certain data scenario it won't consume/take down the master role. In such a case having multiple instances won't help because, presumably, both instances are working off the same source data which would trigger the same bug in all instances. – Entitle

In case of one of the sub-roles fails, everything fail, and then we let Azure to fully restart the role. Of course the roles are ready to recover from any recoverable exception, but in case of a fatal one, everything should fail like a block. – Ellingston

Ok, if that's what makes sense for your system then there's no problems! :) I just wanted to raise the point for discussion so people who read this thread down the road stop and think about it. – Entitle

Answer 4

Why would a 'multi role' be a mess? You could write each worker role implementation as a loosely coupled component and then compose a Worker Role from all appropriate components.

When you later need to separate some of the responsibilities out to a separate worker role, you can compose a new worker role with only this component, while at the same time removing it from the old worker role.

If you wanted to, you could employ late binding so that this could even be done without recompilation, but often I don't think that would be worth the effort.

Calder

The idea is to develop something that can be decouple easily. For example, create all the roles by separate, and use a master worker role to make them work together, but... when the load level be acceptable, put each one to work on its own without code anything. – Ellingston