async/await and opening a FileStream?

Question

I came across the following question when trying to determine if I was using the Stream methods such as ReadAsync and CopyToAsync correctly: C# 4.5 file read performance sync vs async

In this question I read the following in the accepted answer:

Most notably, your "async" test does not use async I/O; with file streams, you have to explicitly open them as asynchronous or else you're just doing synchronous operations on a background thread.

In his asynchronous IO code he was using the following to open the FileStream 'asynchronously':

var file = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, true)

So I was wondering if you intend to use methods such as CopyToAsync whether you should open the underlying FileStream as shown above?, as opposed to doing something simple like the following:

File.Open(filename, FileMode.Open)

Which is how the example in the actual documentation for CopyToAsync demonstrates opening the underlying FileStream: https://msdn.microsoft.com/en-us/library/hh159084(v=vs.110).aspx

If it does not matter which way the underlying FileStream is opened, what does the useAsync parameter of the FileStream constructor do?

Gariepy

Answer 1

So I was wondering if you intend to use methods such as CopyToAsync whether you should open the underlying FileStream as shown above?

Yes. The reason is mostly historical.

First, on Windows, HANDLEs (including file handles) must be opened/created explicitly with an asynchronous flag if you want to do asynchronous (OVERLAPPED) operations on them.

However, the old Windows 95/98/ME line only supported asynchronous operations on serial port and IOCTL (device driver) handles. Asynchronous I/O on disk files wasn't supported on that platform line. And the original .NET did support 98/ME, so the original FileStream just used synchronous I/O. I think (but am not absolutely sure) that APM methods (like FileStream.BeginRead) on Win98/ME were probably just implemented using the so-called "asynchronous delegates" (which just execute a synchronous method like FileStream.Read on a thread pool thread).

So, that's the historical reason why file stream handles were not opened with the asynchronous flag by default.

Which is how the example in the actual documentation for CopyToAsync demonstrates

Unfortunately, a lot of the MSDN examples are rather poor quality. They're OK if you approach them from the perspective of "here's an example of how to call this specific method", but not so great from the perspective of "here's an example of production-quality code that uses this method".

Lansing

Answer 2

So I was wondering if you intend to use methods such as CopyToAsync whether you should open the underlying FileStream as shown above, as opposed to doing something simple like File.Open?

I used ILSpy to decompile and look at File.Open.

public static FileStream Open(string path, FileMode mode)
{
    return File.Open(path, 
                     mode, 
                     (mode == FileMode.Append) 
                         ? FileAccess.Write 
                         : FileAccess.ReadWrite, 
                     FileShare.None);
}

Which calls this:

public static FileStream Open(string path, FileMode mode, FileAccess access, FileShare share)
{
    return new FileStream(path, mode, access, share);
}

And this specific FileStream constructor passes in false for the useAsync parameter. So, yes it appears to matter. However, you can still invoke the async APIs and it will still work as you'd expect.

As Hans Passant states:

The underlying CreateFile() call then uses the FILE_FLAG_OVERLAPPED option. This enables overlapped I/O, a mechanism that enables asynchronous reads and writes at the winapi level.

The FileStream class has an _isAsync bool, and it means "If async IO is not supported on this platform or if this FileStream was not opened with FileOptions.Asynchronous.".

Again, you still get a Task that represents that asynchronous operation as you desire.

Repertoire

Why use a decompiler when you can actually look at the source? – Libretto

Reference source is great and I do use it all the time, but what guarantee is there it's the update to date? I don't actually know... either way leads to an answer. – Repertoire

It says it shows the source for .Net Framework 4.6.1, that should be recent enough. – Libretto

So if I understand, calling the Async API methods will cause the underlying stream to be created with the correct flags even if the FileStream was not constructed with useAsync true? Because it sounds like I should be ensuring the FileStream is created using the FILE_FLAG_OVERLAPPED option. – Adenosine

@AlexHopeO'Connor, no. It will not open it with that flag -- but it will still be async. The APIs still return the Task and it just works. – Repertoire

Answer 3

The MSDN website says:

useAsync

Type: System.Boolean

Specifies whether to use asynchronous I/O or synchronous I/O. However, note that the underlying operating system might not support asynchronous I/O, so when specifying true, the handle might be opened synchronously depending on the platform. When opened asynchronously, the BeginRead and BeginWrite methods perform better on large reads or writes, but they might be much slower for small reads or writes. If the application is designed to take advantage of asynchronous I/O, set the useAsync parameter to true. Using asynchronous I/O correctly can speed up applications by as much as a factor of 10, but using it without redesigning the application for asynchronous I/O can decrease performance by as much as a factor of 10.

Haematite

So it does not matter for other asynchronous IO methods like CopyToAsync? – Adenosine

Yes,matter for other asynchronous IO methods like CopyToAsync. but Unfortunately ,MSDN saying does not matter. – Haematite

I tried this also, the result was on Win 7 x64 SP2, ALWAYS freezes at 80-99% with an exception like: "There couldn't be read any more bytes from the stream". After turning the useAsync flag off everything works like a charm, no freezes anymore. So it seems even MS doesn't really know what that flag does. ;) – Enthrall