.Net framework to manage background running processess on seperate machines

Question

I am having an asp.mvc application which resides on a server.From this application, I want to start a process which is a bit long-running operation and will be resource intensive operation.

So what I want to do is I want to have some user agent like 3 which will be there on 3 machines and this user agent will use resources of their respective machines only.

Like in Hadoop we have master nodes and cluster in which tasks are run on the individual cluster and there is 1 master node keeping track of all those clusters.

In Azure, we have virtual machines on which tasks are run and if require Azure can automatically scale horizontally by spinning up the new instance in order to speed up the task.

So I want to create infrastructure like this where I can submit my task to 3 user agents from the mvc application and my application will keep track of this agents like which agent is free, which is occupied, which is not working something like this.

I would like to receive progress from each of this user agent and show on my MVC application.

Is there any framework in .net from which I can manage this background running operations(tracking, start, stop etc..) or what should be the approach for this?

Update : I don't want to put loads of server for this long running operations and moreover I want to keep track of this long running process too like what they are doing, where is error etc.

Following are the approach which I am thinking and I don't know which will make more sense:

1) Install Windows Service in the form of agents of 2-3 computer on premises to take advantage of resp resources and open a tcp/ip connection with this agents unless and until the long running process is complete.

2) Use hangfire to run this long running process outside of IIS thread but I guess this will put load on server.

I would like to know possible problems of above approaches and if there are any better approaches than this.

Objurgate

That's quite a broad question. I'd personally use some message queue (like rabbitmq) for this - from MVC you just post message describing task to be done and don't care about how much workers there are, and where. Workers listed to queue and rabbitmq will dispatch messages in round-robin manner to them. – Justiceship

@Justiceship I dont want to put this processing load on my server.I want to do this long processing in client environment freeing up my server.My server(where i have hosted my mvc app) would be just responsible for keeping track(job progress,completed,stop etc..) of this long running process.I am thinking to have a wcf service which will be called by mvc app and will be free.This wcf service will be in client environment.So rabbitqueue will be in wcf? – Objurgate

RabbitMQ is message bus. You post message from one place (asp.net service) and receive it in another (one of the workers which are located on different servers). That way you can add\remove workers without touching mvc service, and you don't need to bother where workers are located (that is - don't need to know ip address or anything else). – Justiceship

@Justiceship How about using hangfire mention in one of the answer? – Objurgate

What exactly do you want to do with the result of the process? since asp.net is REST, how are you supposed to show the result (Signalr? separate website? etc?) – Ellette

Answer 1

Hangfire is really a great solution for processing background tasks, and we have used used it extensively in our projects.

We have setup our MVC application on separate IIS servers which is also a hangfire client and just enqueues the jobs needs to be executed by hangfire server. Then we have two hangfire server instances, which are windows service application. So effectively there is no load on the MVC app server to process the background jobs, as it is being processed by separate hangfire servers.

One of the extremely helpful feature of hangfire is its out of the box dashboard, that allows you to monitor and control any aspect of background job processing, including statistics, background job history etc.

Configure the hangfire in application as well as in hangfire servers

public void Configuration(IAppBuilder app)
{
    GlobalConfiguration.Configuration.UseSqlServerStorage("<connection string or its name>");

    app.UseHangfireDashboard();
    app.UseHangfireServer();
}

Please note that you use the same connection string across. Use app.UseHangfireServer() only if you want to use the instance as hangfire server, so in your case you would like to omit this line from application server configuration and use only in the hangfire servers. Also use app.UseHangfireDashboard() in instance which will serve your hangfire dashboard, which would be probably your MVC application.

At that time we have done it using Windows Service, but if had to do it now, I would like to go with Azure worker role or even better now Azure Web Jobs to host my hangfire server, and manage things like auto scaling easily.

Do refer hangfire overview and documentation for more details.

Irmine

Thank you so much for the answer and for your kind efforts towards helping me.I will have same thing like 1 mvc app and 1or2 windows service where background processing will take place.So i have to write app.UseHangfireDashboard(); in both mvc app as well as windows service app? – Objurgate

@Learning-Overthinker-Confused - Use dashboard only in MVC app. And make sure connection string is same for both application server and hangfire server(s) – Irmine

Sorry i mean to ask about app.UseHangfireServer();I will need app.UseHangfireServer(); only in windows service where background job processing will take place and not in mvc(where i will only enqueue jobs).right? – Objurgate

@Learning-Overthinker-Confused - Thats right. In case you have more than 1 HF server, than use app.UseHangfireServer() in all of them. – Irmine

So hangfire server will reside where my background processing will be taking place as i want to process background processing in windows app so hangfire server will reside on windows service.I dont need hangfire server in mvc app because in mvc app i just want to enqueue jobs.Please correct me if i am wrong and sorry if i am asking silly questions but i am just trying to understand things with hangfire :) – Objurgate

@Learning-Overthinker-Confused - No problem. Yeah thats correct. – Irmine

Let us continue this discussion in chat. – Objurgate

Answer 2

Push messages to MSMQ from your MVC app and have your windows services listen (or loop) on new messages entering the queue.

In your MVC app create a ID per message queued, so make restful API calls from your windows services back to the mvc app as you make progress on the job?

Copacetic

Answer 3

Have a look at Hangfire, this can manage background tasks and works across VMs without conflict. We have replaced windows services using this and it works well.

https://www.hangfire.io

Im

So consider i have an mvc application from where i will start long running operation and 1 wcf service where i want to process this long running operation and also this mvc app will display tracking of this long running process.Wcf service will reside in client environment.So if i will fire and forget job from mvc app so this will put load on my mvc app server right? – Objurgate

Yes, Hangfire will create processes in IIS, on the app server. – Im

But i have read that hangfire uses threads outside of IIS from the documentation – Objurgate

OK that is even better, in that case the load will still be on the server but not on IIS or your web app directly. – Im

But i would like to take away that load from the mvc app server and instead put that load on the service which is executing that long running operation.Is this possible with hangfire? – Objurgate

I have only used Hangfire to process background job on the same server as the web app, which means the load is balanced and no single point of failure. I am not sure if Hangfire can be setup to create jobs on one server and then process on another. ..not sure ion that is configurable. Alternatively if you wanted multiple windows services on multiple servers processing the same jobs, you would need to use a queuing mechanism where the service will take a job, set a status and prevent other services from taking the same jobs, you could track each service on each machine on that case. – Im

Let us continue this discussion in chat. – Objurgate

Answer 4

Give a try to http://easynetq.com/

EasyNetQ is a simple to use, opinionated, .NET API for RabbitMQ.

EasyNetQ is a collection of components that provide services on top of the RabbitMQ.Client library. These do things like serialization, error handling, thread marshalling, connection management, etc.

To publish with EasyNetQ

var message = new MyMessage { Text = "Hello Rabbit" };
bus.Publish(message);

To subscribe to a message we need to give EasyNetQ an action to perform whenever a message arrives. We do this by passing subscribe a delegate:

bus.Subscribe<MyMessage>("my_subscription_id", msg => Console.WriteLine(msg.Text));

Now every time that an instance of MyMessage is published, EasyNetQ will call our delegate and print the message’s Text property to the console.

The performance of EasyNetQ is directly related to the performance of the RabbitMQ broker. This can vary with network and server performance. In tests on a developer machine with a local instance of RabbitMQ, sustained over-night performance of around 5000 2K messages per second was achieved. Memory use for all the EasyNetQ endpoints was stable for the overnight run

Dinothere

But i need to install RabbitMQ on server right in order to use RabbitMQ? – Objurgate

Yeah, as easyNetQ is just an API for RabbitMQ, Though it will handle all the complexity of RabbitMQ, and making AMQP smoother. – Dinothere

Lets say i submit 2 jobs in RabbitMq queue and i want to track progress for each of this job.Is this possible with RabbitMQ and with easyNetQ? – Objurgate

you can achieve it by delivery tags, You may review this link, it has very nice explanation : jack-vanlightly.com/blog/2017/3/11/… – Dinothere

Thank you so much for the answer and for your kind help towards helping me.Consider i am having an mvc app from where i will publish my jobs which resides in azure cloud.I am having windows service which is in client premises performing all those background jobs.So where does this rabbitmq will reside?On azure or on client premises in order to handshake with both producer and consumer – Objurgate