Execute certain background Tasks in separate ThreadPool to avoid starvation to critical Tasks executed in main thread

Question

Execute certain background Tasks (not threads) in separate ThreadPool to avoid starvation to critical Tasks (not threads) executed in main thread

Our Scenario

We host a high-volume WCF web service, which logically has the following code:

void WcfApiMethod()
{
   // logic

   // invoke other tasks which are critical
   var mainTask = Task.Factory.StartNew(() => { /* important task */ });
   mainTask.Wait();

   // invoke background task which is not critical
   var backgroundTask = Task.Factory.StartNew(() => { /* some low-priority background action (not entirely async) */ });
   // no need to wait, as this task is best effort. Fire and forget

   // other logic
}

// other APIs

Now, the issue, in certain scenarios, the low-priority background task may take longer (~ 30 sec), for e.g., to detect SQL connection issue, DB perf issues, redis cache issues, etc. which will make those background threads delayed, which means TOTAL PENDING TASK COUNT will increase, due to high volume.

This creates a scenario where, newer executions of the API cannot schedule high-priority task, because, lot of background tasks are in queue.

Solutions we tried

1. Adding TaskCreationOptions.LongRunning to high-pri task will immediately execute it. However, this cannot be a solution for us, as there are lot of tasks being invoked everywhere in the system, we cannot make them long-running everywhere. Also, WCF handling of incoming APIs would rely on .NET thread pool, which is in starvation now.

2. Short-circuit low-pri-background task creation, via Semaphore. Only spawn threads if the system has capacity to process them (check if earlier created threads have exited). If not, just don't spawn up threads. For e.g., due to an issue (say DB perf issue), ~ 10,000 background threads (non-async) are on IO wait, which may cause thread-starvation in main .net thread pool. In this specific case, we could add a Semaphore to limit creation to 100, so if 100 tasks are stuck, 101st task won't be created in the first place.

Ask on alternative solution

Is there a way to specifically spawn "tasks" on "custom threads/ thread pool", instead of the default .NET thread pool. This is for the background tasks I mentioned, so in case they get delayed, they don't bring down the whole system with them. May be override and create a custom TaskScheduler to be passed into Task.Factory.StartNew() so, tasks created would NOT be on default .NET Thread Pool, rather some other custom pool.

Alfaro

Have you considered the option of scheduling low priority tasks through a limited concurrency TaskScheduler? – Mercuri

I was looking for an inbuilt .net solution. Your tag "limited concurrency TaskScheduler" led me to Johnbot's comment codereview.stackexchange.com/questions/203213/… "Take a look at ConcurrentExclusiveSchedulerPair or more specifically ConcurrentExclusiveSchedulerPair.ConcurrentScheduler which is the built-in way of doing it. – Johnbot Aug 1 '19 at 9:22" Tested, this is sufficient. Thank you. – Alfaro

Answer 1

Here is a static RunLowPriority method you could use in place of Task.Run. It has overloads for simple and generic tasks, and for normal and asynchronous delegates.

const int LOW_PRIORITY_CONCURRENCY_LEVEL = 2;
static TaskScheduler LowPriorityScheduler = new ConcurrentExclusiveSchedulerPair(
    TaskScheduler.Default, LOW_PRIORITY_CONCURRENCY_LEVEL).ConcurrentScheduler;

public static Task RunLowPriority(Action action,
    CancellationToken cancellationToken = default)
{
    return Task.Factory.StartNew(action, cancellationToken,
        TaskCreationOptions.DenyChildAttach, LowPriorityScheduler);
}

public static Task RunLowPriority(Func<Task> function,
    CancellationToken cancellationToken = default)
{
    return Task.Factory.StartNew(function, cancellationToken,
        TaskCreationOptions.DenyChildAttach, LowPriorityScheduler).Unwrap();
}

public static Task<TResult> RunLowPriority<TResult>(Func<TResult> function,
    CancellationToken cancellationToken = default)
{
    return Task.Factory.StartNew(function, cancellationToken,
        TaskCreationOptions.DenyChildAttach, LowPriorityScheduler);
}

public static Task<TResult> RunLowPriority<TResult>(Func<Task<TResult>> function,
    CancellationToken cancellationToken = default)
{
    return Task.Factory.StartNew(function, cancellationToken,
        TaskCreationOptions.DenyChildAttach, LowPriorityScheduler).Unwrap();
}

The actions scheduled through the RunLowPriority method will run on ThreadPool threads, but at maximum 2 of all the available ThreadPool threads can be concurrently assigned to RunLowPriority tasks.

Keep in mind that the Elapsed event of a System.Timers.Timer that has its SynchronizingObject property set to null runs in ThreadPool threads too. So if you are doing low priority work inside this handler, you should probably schedule it through the same limited concurrency scheduler:

var timer = new System.Timers.Timer();
timer.Elapsed += (object sender, System.Timers.ElapsedEventArgs e) =>
{
    Thread.Sleep(10); // High priority code
    var fireAndForget = RunLowPriority(() =>
    {
        if (!timer.Enabled) return;
        Thread.Sleep(1000); // Simulate long running code that has low priority
    });
};

Mercuri

why DenyChildAttach? – Alfaro

@Abin to make it equivalent to Task.Run. You can read here about the similarities and differences between Task.Run and Task.Factory.StartNew. – Mercuri

Answer 2

Based on https://codereview.stackexchange.com/questions/203213/custom-taskscheduler-limited-concurrency-level?newreg=acb8e97fe4c94844a660bcd7473c4876, there does exist an inbuilt solution to limit thread-spawns via limited concurrency TaskScheduler.

Inbuilt ConcurrentExclusiveSchedulerPair.ConcurrentScheduler could be used to achieve this.

For the above scenario, following code limits background threads from wrecking the application/ prevents thread-starvation.

        {
            // fire and forget background task
            var task = Task.Factory.StartNew(
                () =>
                {
                    // background threads
                }
                , CancellationToken.None
                , TaskCreationOptions.None
                , concurrentSchedulerPair.ConcurrentScheduler);
        }

        private static ConcurrentExclusiveSchedulerPair concurrentSchedulerPair = new ConcurrentExclusiveSchedulerPair(
            TaskScheduler.Default,
            maxConcurrencyLevel: 100);

A caution on using TaskScheduler.Default and maxConcurrencyLevel:100 parameters, say, you create 10000 tasks using this limited-conc-scheduler and try to immediately spawn another thread using 'default-scheduler', that new spawn would be blocked unless all 100 threads are created. If you try maxConcurrencyLevel:10, new thread spawns are immediately and not blocking once all 10 threads are instantiated.

Thanks @Theodor Zoulias for the pointer.

Alfaro