Is it possible to "await yield return DoSomethingAsync()"

Question

Are regular iterator blocks (i.e. "yield return") incompatible with "async" and "await"?

This gives a good idea of what I'm trying to do:

async Task<IEnumerable<Foo>> Method(String [] Strs)
{
    // I want to compose the single result to the final result, so I use the SelectMany
    var finalResult = UrlStrings.SelectMany(link =>   //i have an Urlstring Collection 
                   await UrlString.DownLoadHtmlAsync()  //download single result; DownLoadHtmlAsync method will Download the url's html code 
              );
     return finalResult;
}

However, I get a compiler error citing "unable to load message string from resources".

Here is another attempt:

async Task<IEnumerable<Foo>> Method(String [] Strs)
{
    foreach(var str in strs)
    {
        yield return await DoSomethingAsync( str)
    }
}

But again, the compiler returns an error: "unable to load message string from resources".

---

Here is the real programming code in my project

This is very useful when I have an List Task,that task can be download HTML from a URL and I use the syntax "yield return await task", the result is I want IEnumerable<Foo>. I don't want write this code:

async Task<IEnumerable<String>> DownLoadAllURL(String [] Strs)
{
    List<Foo> htmls= new ...
    foreach(var str in strs)
    {
        var html= await DownLoadHtmlAsync( str)
        htmls.Add(item)
    }
    return htmls;
}

But it seems that I have to.

Thanks for any help.

Boxwood

This question isn't particularly clear, you should elaborate on what it is exactly that you wish to do - its difficult to determine what it is you wish to do from just a code sample alone. – Kazue

The Ix_Experimental-Async NuGet package includes "asynchronous enumerators" complete with LINQ support. They use the IAsyncEnumerator<T> type as defined by Arne. – Cablegram

@StephenCleary that package has been delisted. Since this was ages ago & authored by MS's rxteam, do you know if that got rolled into RX? – Undervalue

@JoeBrockhaus: Looks like it lives on as Ix-Async. – Cablegram

Answer 1

What you are describing can be accomplished with the Task.WhenAll method. Notice how the code turns into a simple one-liner. What happens is that each individual url begins downloading and then WhenAll is used combine those operations into a single Task which can be awaited.

Task<IEnumerable<string>> DownLoadAllUrls(string[] urls)
{
    return Task.WhenAll(from url in urls select DownloadHtmlAsync(url));
}

Nordin

async/await are not needed in this case. Just remove async from the method and do return Task.WhenAll directly. – Richart

@luiscubal: Yep, fixed. – Nordin

The last line can be written more succinctly as urls.Select(DownloadHtmlAsync) – Flats

This turned out to be the exact problem I was facing (lazily downloading strings from a series of URIs). Thank you. – Vinaigrette

This doesn't actually answer the question of Is it possible to await yield? You just found a work-around for this one use-case. Didn't answer the general question. – Bibcock

Have you actually tested this code in an IDE? I get Cannot implicitly convert type 'System.Threading.Tasks.Task<string[]>' to 'System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<string>>'. – Malebranche

A succinct solution if the result doesn't need to be an IEnumerable is just Task<string[]> DownloadAllUrls(string[] urls) => Task.WhenAll(urls.Select(DownloadHtmlAsync)); – Malebranche

I would be inclined to return Task<string[]> as this would indicate that you are no longer returning an iterator with deferred execution ie. all of the URLs are being downloaded. – Stagemanage

For a 2022 update : https://mcmap.net/q/157514/-is-it-possible-to-quot-await-yield-return-dosomethingasync-quot – Sir

Answer 2

tl;dr Iterators as implemented with yield are a blocking construct, so as of right now await and yield are incompatible.

Long Because iterating over an IEnumerable is a blocking operation, calling a method marked as async will still execute it in a blocking manner, since it has to wait for that operation to finish.

async Task<IEnumerable<Foo>> Method(String [] Strs)
{
  foreach(var str in strs)
  {
    yield return await DoSomethingAsync( str)
  }
}  

The awaiting Method mixes meanings. Do you want to wait until the Task has an IEnumerable and then block on iterating over it? Or are you trying to await each value of the IEnumerable?

I assume the second is the desired behavior and in that case the existing Iterator semantics will not work. The IEnumerator<T> interface is basically

public interface IEnumerator<T>
  T Current;
  bool MoveNext();
}

I'm ignoring Reset() since it makes no sense for a sequence of asynchronous results. But what you would need is something like this:

public interface IAsyncEnumerator<T>
  T Current;
  Task<bool> MoveNext();
}

Of course, foreach also won't work with this and you'd have to iterate manually like this:

var moveNext = await asyncEnumerator.MoveNext();
while(moveNext) {

  // get the value that was fetche asynchronously
  var v = asyncEnumerator.Current;

  // do something with that value

  // suspend current execution context until next value arrives or we are done
  moveNext = await asyncEnumerator.MoveNext();
}

Boss

Do you think it's possible that the next C# language version will add foreach support for your hypothetical IAsyncEnumerator<T>? – Nutty

@Nutty I think its very much in the pipeline. for C# 8.0/ Had read about it. – Aedile

@Nutty , learn.microsoft.com/en-us/dotnet/csharp/whats-new/… . – Spoilfive

Answer 3

According to the new features at C# 8.0 (link#1 and link#2) we'll have IAsyncEnumerable<T> interface support that will allow to implement your second attempt. It will look like this:

async Task<Foo> DoSomethingAsync(string url)
{
    ...
}       
// producing IAsyncEnumerable<T>
async IAsyncEnumerable<Foo> DownLoadAllURL(string[] strs)
{
    foreach (string url in strs)
    {
        yield return await DoSomethingAsync(url);
    }
}
...
// using
await foreach (Foo foo in DownLoadAllURL(new string[] { "url1", "url2" }))
{
    Use(foo);
}

We can achieve the same behavior at C# 5 but with a different semantics:

async Task<Foo> DoSomethingAsync(string url)
{
    ...
}
IEnumerable<Task<Foo>> DownLoadAllURL(string[] strs)
{
    foreach (string url in strs)
    {
        yield return DoSomethingAsync(url);
    }
}

// using
foreach (Task<Foo> task in DownLoadAllURL(new string[] { "url1", "url2" }))
{
    Foo foo = await task;
    Use(foo);
}

Brian Gideon's answer implies that the calling code will get asynchronously a collection of results that were obtained in parallel. The code above implies that the calling code will get results like from a stream one by one in asynchronous manner.

Turnage

This should be the new accepted answer, IAsyncEnumerable works for me – Sir

Answer 4

I know that I'm too late with the answer, but here is another simple solution that can be achieved with this library:
GitHub: https://github.com/tyrotoxin/AsyncEnumerable
NuGet.org: https://www.nuget.org/packages/AsyncEnumerator/
It's much simpler than Rx.

using System.Collections.Async;

static IAsyncEnumerable<string> ProduceItems(string[] urls)
{
  return new AsyncEnumerable<string>(async yield => {
    foreach (var url in urls) {
      var html = await UrlString.DownLoadHtmlAsync(url);
      await yield.ReturnAsync(html);
    }
  });
}

static async Task ConsumeItemsAsync(string[] urls)
{
  await ProduceItems(urls).ForEachAsync(async html => {
    await Console.Out.WriteLineAsync(html);
  });
}

Teressaterete

Answer 5

This feature will be available as of C# 8.0. https://blogs.msdn.microsoft.com/dotnet/2018/11/12/building-c-8-0/

From MSDN

Async streams

The async/await feature of C# 5.0 lets you consume (and produce) asynchronous results in straightforward code, without callbacks:

async Task<int> GetBigResultAsync()
{
    var result = await GetResultAsync();
    if (result > 20) return result; 
    else return -1;
}

It is not so helpful if you want to consume (or produce) continuous streams of results, such as you might get from an IoT device or a cloud service. Async streams are there for that.

We introduce IAsyncEnumerable, which is exactly what you’d expect; an asynchronous version of IEnumerable. The language lets you await foreach over these to consume their elements, and yield return to them to produce elements.

async IAsyncEnumerable<int> GetBigResultsAsync()
{
    await foreach (var result in GetResultsAsync())
    {
        if (result > 20) yield return result; 
    }
}

Hilbert

Answer 6

There was a plan to do

https://github.com/dotnet/csharplang/issues/43

But currently it not possible

Updated: It was added in C# 8.0

Sotted

Answer 7

First of all, keep in mind that the Async stuff is not finished. The C# team still has a long way to go before C# 5 is released.

That being said, I think you may want to gather the tasks that are being fired off in the DownloadAllHtml function in a different way.

For example, you can use something like this:

IEnumerable<Task<string>> DownloadAllUrl(string[] urls)
{
    foreach(var url in urls)
    {
        yield return DownloadHtmlAsync(url);
    }
}

async Task<string> DownloadHtmlAsync(url)
{
    // Do your downloading here...
}

Not that the DownloadAllUrl function is NOT an async call. But, you can have the async call implemented on another function (i.e. DownloadHtmlAsync).

The Task Parallel Library has the .ContinueWhenAny and .ContinueWhenAll functions.

That can be used like this:

var tasks = DownloadAllUrl(...);
var tasksArray = tasks.ToArray();
var continuation = Task.Factory.ContinueWhenAll(tasksArray, completedTasks =>
{
    completedtask
});
continuation.RunSynchronously();

Diagnostics

You can also have a 'header' (before the loop) if you define your method as async Task<IEnumerable<Task<string>>> DownloadAllUrl. Or, if you want 'footer' actions IEnumerable<Task>. E.g. gist.github.com/1184435 – Messinger

Now that the async stuff is finished and C# 5.0 is released, this can be updated. – Stairs

I like this one, but how in this case foreach(var url in await GetUrls()) { yield return GetContent(url) }; I only found a way splitting in two methods. – Nikolaus

The interesting thing about DownloadAllUrl method is that it is not quite correct to say that it isn't an async call. The thing that stops it from immediately running asynchronously is actually the yield keyword. As long you iterate through the returned IEnumerable, all the tasks will be started. Whether the tasks are being awaited on is another issue. – Anthracosilicosis

Answer 8

Yield does not work with await, unfortunately. But this is what Rx is for. Check out https://msdn.microsoft.com/library/hh242985

Ptarmigan

Answer 9

This solution works as expected. Note the await Task.Run(() => enumerator.MoveNext()) part.

using (var enumerator = myEnumerable.GetEnumerator())
{
    while (true)
    {
        if (enumerator.Current != null)
        {
            //TODO: do something with enumerator.Current
        }

        var enumeratorClone = monitorsEnumerator;
        var hasNext = await Task.Run(() => enumeratorClone.MoveNext());
        if (!hasNext)
        {
            break;
        }
    }
}

Dead