Should we use ConfigureAwait(false) in libraries that call async callbacks?

Question

There are lots of guidelines for when to use ConfigureAwait(false), when using await/async in C#.

It seems the general recommendation is to use ConfigureAwait(false) in library code, as it rarely depends on the synchronization context.

However, assume we are writing some very generic utility code, which takes a function as input. A simple example could be the following (incomplete) functional combinators, to make simple task-based operations easier:

Map:

public static async Task<TResult> Map<T, TResult>(this Task<T> task, Func<T, TResult> mapping)
{
    return mapping(await task);
}

FlatMap:

public static async Task<TResult> FlatMap<T, TResult>(this Task<T> task, Func<T, Task<TResult>> mapping)
{
    return await mapping(await task);
}

The question is, should we use ConfigureAwait(false) in this case? I am unsure how the context capture works wrt. closures.

On one hand, if the combinators are used in a functional way, the synchronization context should not be necessary. On the other hand, people might misuse the API, and do context dependent stuff in the provided functions.

One option would be to have separate methods for each scenario (Map and MapWithContextCapture or something), but it feels ugly.

Another option might be to add the option to map/flatmap from and into a ConfiguredTaskAwaitable<T>, but as awaitables don't have to implement an interface this would result in a lot of redundant code, and in my opinion be even worse.

Is there a good way to switch the responsibility to the caller, such that the implemented library doesn't need to make any assumptions on whether or not the context is needed in the provided mapping-functions?

Or is it simply a fact, that async methods don't compose too well, without various assumptions?

EDIT

Just to clarify a few things:

1. The problem does exist. When you execute the "callback" inside the utility function, the addition of ConfigureAwait(false) will result in a null sync. context.
2. The main question is how we should tackle the situation. Should we ignore the fact that someone might want to use the sync. context, or is there a good way to shift the responsibility out to the caller, apart from adding some overload, flag or the like?

As a few answers mention, it would be possible to add a bool-flag to the method, but as I see it, this is not too pretty either, as it will have to be propagated all the way through the API's (as there are more "utility" functions, depending on the ones shown above).

Fabricate

If you want to know what the current context is in the delegate when using ConfigureAwait(false) all you had to do was run the code once with a delegate that prints out the current context, or even just code that would crash if the original context wasn't captured. It'd have taken you far less time than writing this question, given that you already have all of the code written out. – Cruzeiro

@Cruzeiro Can't argue that. Still, seeing whether or not the problem exists, doesn't necessarily entail that a solution is easily found. And sharing the information doesn't hurt, I hope. – Fabricate

Rather than saying, "I'm not sure if this problem exists, but if it does, how do I solve it?" spend the 30 seconds to figure out if it exists, and then ask, "how do I solve this problem that I've found?" (or don't ask a question at all because the problem doesn't exist, based on whichever is the case). – Cruzeiro

@Cruzeiro I certainly have made tests to ensure that there "was a problem". The question is how to handle it, or if we should handle it at all. If the wording doesn't tell that I am interested in some more detailed information, it is a mistake from my side. – Fabricate

The question, as I read it, seems to be asking if adding ConfigureAwait(false) to your method will result in the current context being null or non-null in the callback. If you know that it's non-null, you should make that clear in the question that you're simply asking about how to best deal with that fact. Specifically, your question states: I am unsure how the context capture works wrt. closures. This indicates you don't know what will happen. If you do know, but don't know how to expose a particular set of functionality given that behavior, then that is where you're being unclear. – Cruzeiro

Related question: #13489565 – Hitormiss

Answer 1

When you say await task.ConfigureAwait(false) you transition to the thread-pool causing mapping to run under a null context as opposed to running under the previous context. That can cause different behavior. So if the caller wrote:

await Map(0, i => { myTextBox.Text = i.ToString(); return 0; }); //contrived...

Then this would crash under the following Map implementation:

var result = await task.ConfigureAwait(false);
return await mapper(result);

But not here:

var result = await task/*.ConfigureAwait(false)*/;
...

Even more hideous:

var result = await task.ConfigureAwait(new Random().Next() % 2 == 0);
...

Flip a coin about the synchronization context! This looks funny but it is not as absurd as it seems. A more realistic example would be:

var result =
  someConfigFlag ? await GetSomeValue<T>() :
  await task.ConfigureAwait(false);

So depending on some external state the synchronization context that the rest of the method runs under can change.

This also can happen with very simple code such as:

await someTask.ConfigureAwait(false);

If someTask is already completed at the point of awaiting it there will be no switch of context (this is good for performance reasons). If a switch is necessary then the rest of the method will resume on the thread pool.

This non-determinism a weakness of the design of await. It's a trade-off in the name of performance.

The most vexing issue here is that when calling the API is is not clear what happens. This is confusing and causes bugs.

What to do?

Alternative 1: You can argue that it is best to ensure deterministic behavior by always using task.ConfigureAwait(false).

The lambda must make sure that it runs under the right context:

var uiScheduler = TaskScheduler.FromCurrentSynchronizationContext;
Map(..., async x => await Task.Factory.StartNew(
        () => { /*access UI*/ },
        CancellationToken.None, TaskCreationOptions.None, uiScheduler));

It's probably best to hide some of this in a utility method.

Alternative 2: You can also argue that the Map function should be agnostic to the synchronization context. It should just leave it alone. The context will then flow into the lambda. Of course, the mere presence of a synchronization context might alter the behavior of Map (not in this particular case but in general). So Map has to be designed to handle that.

Alternative 3: You can inject a boolean parameter into Map that specifies whether to flow the context or not. That would make the behavior explicit. This is sound API design but it clutters the API. It seems inappropriate to concern a basic API such as Map with synchronization context issues.

Which route to take? I think it depends on the concrete case. For example, if Map is a UI helper function it makes sense to flow the context. If it is a library function (such as a retry helper) I'm not sure. I can see all alternatives make sense. Normally, it is recommended to apply ConfigureAwait(false) in all library code. Should we make an exception in those cases where we call user callbacks? What if we have already left the right context e.g.:

void LibraryFunctionAsync(Func<Task> callback)
{
    await SomethingAsync().ConfigureAwait(false); //Drops the context (non-deterministically)
    await callback(); //Cannot flow context.
}

So unfortunately, there is no easy answer.

Bibliopole

Usr, in the light of this related question, do you think it might be a good idea to not use ConfigureAwait at all (especially if the code runs in ASP.NET) and leave it up to the client (the top-level caller) to use Task.Runor something like WithNoContext to control the context for the whole chain of async calls? – Hols

@Noseratio that would make Map a more special-purpose function because it now deals with the synccontext in addition to doing its business. If that's architecturally OK that is a valid route. I don't think that's pretty, though. I hate hidden dependencies on hidden state. The presence of a synccontext is not transparent. It changes semantics meaningfully. – Bibliopole

Usr, I see your point and I tend to agree, although one might argue that general-purpose libraries are usually context-agnostic, i.e., they shouldn't depend on or alter the current context. – Hols

@Hols My thinking on this has evolved. I tend to agree with you now. You might read the edit if you care. – Bibliopole

A nice amendment, @usr! I myself stopped using ConfigureAwait(false) in the libraries. Then if I call an API and I feel it might be performance-sensitive to the context, I just wrap it with Task.Run. It seems though, these days no one cares about how to squeeze out an extra bit of performance with tweaks like ConfigureAwait. People just use JavaScript for both front-end and back-end, and nobody seems to care about how it works behind the scene. Apparently, it's much easier/cheaper just to throw more virtualized hardware into it if needed. – Hols

Answer 2

The question is, should we use ConfigureAwait(false) in this case?

Yes, you should. If the inner Task being awaited is context aware and does use a given synchronization context, it would still be able to capture it even if whoever is invoking it is using ConfigureAwait(false). Don't forget that when disregarding the context, you're doing so in the higher level call, not inside the provided delegate. The delegate being executed inside the Task, if needed, will need to be context aware.

You, the invoker, have no interest in the context, so it's absolutely fine to invoke it with ConfigureAwait(false). This effectively does what you want, it leaves the choice of whether the internal delegate will include the sync context up to the caller of your Map method.

Edit:

The important thing to note is that once you use ConfigureAwait(false), any method execution after that would be on on an arbitrary threadpool thread.

A good idea suggested by @i3arnon would be to accept an optional bool flag indicating whether context is needed or not. Although a bit ugly, would be a nice work around.

Girlfriend

The lambda will capture the context at the point where it is called. At that point the sync context might be null already. – Bibliopole

But the user will be providing him that delegate. If he needs to context, he should capture it. – Girlfriend

I must have misunderstood you. – Bibliopole

@Bibliopole If you call Map and provide it with a task returning delegate that is context aware, it would still be context aware even if I wrap it with await providedTask.ConfigureAwait(false) – Girlfriend

@YuvalItzchakov But the point is that this forces the delegate to be explicitly aware of the context that it wishes to use upon creation of the delegate, rather than relying on the current context when invoked. Since the former requires explicit work, and the latter is implicit in the workings of await, it's very much reasonable to go out of one's way to support the latter. Just because you can make it work doesn't mean you should have to. It removes much of the utility of such a method. – Cruzeiro

It turns out I even misunderstood the question. I thought he wants to call ConfigureAwait(false) on everything else in his util function. That would transition to the thread-pool (potentially non-deterministically) and call the delegate under different contexts. That would be a good question. Now calling ConfigureAwait on the resulting task is a much more mundane question... – Bibliopole

@Bibliopole You didn't get me wrong it seems. I am talking about calling ConfigureAwait(false) inside the utility function itself, and not on the result of a call to the utility function. – Fabricate

@Fabricate yes. When you say await task.CA(false) you transition to the thread-pool causing mapping to run under a null context as opposed to running under the previous context. Is that the question? – Bibliopole

@Bibliopole You're causing the continuation to execute on a threadpool thread. – Girlfriend

@Bibliopole Yes, it is exactly my question in some sense. The more interesting aspect, of course, is if there is a good way to push the responsibility to the caller in some fancy way. – Fabricate

@Fabricate My response talks about calling CA inside tge utility function, not inside the delegate provided to the Task – Girlfriend

No, the delegate itself runs on the TP. Make the delegate async _ => { cw(threadid); } to see that. – Bibliopole

Which delegate? The one encapsulated inside the task? – Girlfriend

@Fabricate You can accept a SynchronizationContext in the method and post to it when it isn't null. This can be an optional parameter with a null default value. Or just a boolean whether to use ConfigureAwait. – Dichromatism

Answer 3

I think the real issue here comes from the fact that you are adding operations to Task while you actually operate on the result of it.

There's no real reason to duplicate these operations for the task as a container instead of keeping them on the task result.

That way you don't need to decide how to await this task in a utility method as that decision stays in the consumer code.

If Map is instead implemented as follows:

public static TResult Map<T, TResult>(this T value, Func<T, TResult> mapping)
{
    return mapping(value);
}

You can easily use it with or without Task.ConfigureAwait accordingly:

var result = await task.ConfigureAwait(false)
var mapped = result.Map(result => Foo(result));

Map here is just an example. The point is what are you manipulating here. If you are manipulating the task, you shouldn't await it and pass the result to a consumer delegate, you can simply add some async logic and your caller can choose whether to use Task.ConfigureAwait or not. If you are operating on the result you don't have a task to worry about.

You can pass a boolean to each of these methods to signify whether you want to continue on the captured context or not (or even more robustly pass an options enum flags to support other await configurations). But that violates separation of concerns, as this doesn't have anything to do with Map (or its equivalent).

Dichromatism

But in that case the method wouldn't make any sense - it would just be a more verbose way of calling the mapping directly. Although the example is simple, I am talking about a rather fundamental way of abstracting over a task. – Fabricate

With all due respect this Map method is useless. It could simply be Foo(await task). – Nonconductor

That would be useful when the method has a single line of execution. If he needs to operate on the returned Task, he'll be forced to await – Girlfriend

@Fabricate there's a difference whether you are operating on the task or on the result. You can do whatever you want on the result without worrying (map is just an example). When you're operating on the task you probably shouldn't accept consumer delegates. – Dichromatism

@SriramSakthivel It's useless in the example itself. That's the point. It shouldn't be defined on a Task. – Dichromatism

@Dichromatism I am not arguing whether or not the code is useful or not. The example is simple, yes, but being able to compose async methods in a functional way certainly is interesting to me. – Fabricate

@Fabricate I'm not talking about the specific example. I'm talking about separation of concerns. Your utility method should either be a "logic" method on that value, or an async utility on the Task. You're troubles come from mixing the two together. – Dichromatism

@Fabricate you can pass a boolean every time to specify how the await should behave (or even more robust, an enum of options) but you don't need to do that to begin with. – Dichromatism

I think this answer misses the point by saying Map is defined wrong. It's a totally contrived example. Of course you wouldn't design the signature like this if the method's only purpose is to act on the result. A more realistic example of passing an async delegate is when you want deferred/lazy execution. i.e. caller defines "what", library method defines "when" or even "if" (and perhaps does something after). That can't be refactored to this. I interpret the question as: "assuming we need to await the delegate's Task in the library method, should we use ConfigureAwait(false)?" – Mini

@ToddMenier Using ConfigureAwait on the delegate's task has no effect on the code inside the delegate. It only affects the library's code after that await. The issue with the design in question is that it requires awaiting both the "extended" task and the async delegate because using ConfigureAwait on the task affects the delegate. That basically makes this method like ContinueWith and so to be done correctly requires all of its expressiveness. – Dichromatism

@Dichromatism That makes perfect sense. After a bit of head-scratching (and rereading of the question), I clearly misinterpreted it. – Mini