Long running task in WebAPI

Question

Here's my problem: I need to call multiple 3rd party methods inside an ApiController. The signature for those methods is Task DoSomethingAsync(SomeClass someData, SomeOtherClass moreData). I want those calls to continue running in the background, after the ApiController has sent the data back to the client. When DoSomethingAsync completes I want to do some logging and maybe save some data to the file system. How can I do that? I'd prefer to use the asyny/await syntax.

Dugan

You task has something like .ContinueWith. this is where you can do your logging. – Manilla

When you say "after the ApiController has sent the data back to the client", do you mean that connection is closed as well? Can a client receive data from server while still keeping connection open? – Ebenezer

Yes, the connection is closed. – Dugan

check out hangfire.io – Franklinfranklinite

For .Net Core 2.x, check out IHostedService. A good example at this blog: stevejgordon.co.uk/asp-net-core-2-ihostedservice. Similar to hangfire concept. – Thrombin

Answer 1

Stephen described why starting essentially long running fire-and-forget tasks inside an ApiController is a bad idea.

Perhaps you should create a separate service to execute those fire-and-forget tasks. That service could be a different ApiController, a worker behind a queue, anything that can be hosted on its own and have an independent lifetime.

This would make management of the different task lifetimes much easier and separate the concerns of the long-running tasks from the ApiController's core responsibilities.

Komsomolsk

Another reason why it is better to have a separate service is to make it easier to scale out. For example, maybe you find that you need more servers to run your long running task, but not more servers to run your website. By splitting these up at the beginning you can better customize your scale-out solution. – Doro

How to implement it? – Realm

@KiranAhir it's explained in the docs: Background tasks with hosted services in ASP.NET Core. In a web app, you can create a BackgroundService to take care of long tasks. You can also create a separate Web API project with a Background service to receive requests from the front-end web app. There's even a Worker Service template that creates a standalone background service – Komsomolsk

Answer 2

Great news, there is a new solution in .NET 4.5.2 called the QueueBackgroundWorkItem API. It's really simple to use:

HostingEnvironment.QueueBackgroundWorkItem(ct => DoSomething(a, b, c));

Here's an article that describes it in detail.

https://blogs.msdn.microsoft.com/webdev/2014/06/04/queuebackgroundworkitem-to-reliably-schedule-and-run-background-processes-in-asp-net/

And here's anohter article that mentions a few other approaches not mentioned in this thread. http://www.hanselman.com/blog/HowToRunBackgroundTasksInASPNET.aspx

Stretchy

QueueBackgroundWorkItem can be a solution if the long-running task is not really that long-running, i.e. if it can finish in less than 90 seconds. If it really is long-running another solution is needed (like handing it over to a Windows service) – Rollway

I couldn't get this to work in a self-hosted WebAPI within a Windows Service (backend data layer). – Keelykeen

QueueBackgroundWorkItem doesn't exist in .NET Core (.NET 5+), fyi. – Wit

Answer 3

You almost never want to do this. It is almost always a big mistake.

ASP.NET (and most other servers) work on the assumption that it's safe to tear down your service once all requests have completed. So you have no guarantee that your logging will be done, or that your data will be written to disk. Particularly with the disk writes, it's entirely possible that your writes will be corrupted.

That said, if you are absolutely sure that you want to implement this extremely dangerous design, you can use the BackgroundTaskManager from my blog.

Update: I've written a blog series that goes into detail on a proper solution for request-extrinsic code. In summary, what you really want to do is move the request-extrinsic code out of ASP.NET. Introduce a durable queue and an independent processor; the ASP.NET controller action will place a request onto the queue, and the independent processor will read requests and execute them. This "processor" can be an Azure Function/WebJob, Win32 Service, etc.

Unstuck

Correct before Azure-WebJobs. Now you can do this with Azure-WebJobs. A worker role is the most robust approach. – Mismanage

I find this is an acceptable trade-of for things like: debug-logging, performance recording, event tracking (using an external service). In all of these cases a "fire-and-forget" semantic is acceptable and we don't really care if individual events may get lost in the case of a service tear down. Or is there something I'm missing? – Beeson

@JohannesRudolph: I would say the primary use case is cache updates. It's an acceptable way to do logging, if you accept that your logs may not have all data. I would interpret "event tracking" as a business requirement, so not a good use case for this. If "event tracking" isn't important, then "debug logging", "performance recording", and "event tracking" are just three different kinds of logging. Note that ASP.NET 4.5.2 has something similar built-in now. – Unstuck

Answer 4

As pointed out by others, it is not recommended. However, whenever there is a need there is a way, so take a look at IRegisteredObject

See also

http://haacked.com/archive/2011/10/16/the-dangers-of-implementing-recurring-background-tasks-in-asp-net.aspx/

Aristocracy

Answer 5

Though the question is several years old, best possible solution now is to use Singal R in this case.

https://github.com/Myrmex/signalr-notify-progress

Escheat