Linux and I/O completion ports?

Question

Using winsock, you can configure sockets or seperate I/O operations to "overlap". This means that calls to perform I/O are returned immediately, while the actual operations are completed asynchronously by separate worker threads.

Winsock also provides "completion ports". From what I understand, a completion port acts as a multiplexer of handles (sockets). A handle can be demultiplexed if it isn't in the middle of an I/O operation, i.e. if all its I/O operations are completed.

So, on to my question... does linux support completion ports or even asynchronous I/O for sockets?

Peavy

epoll is Linux's fastest method for writing servers. Note that Linux currently has a greater share of the Internet server market than Windows. – Lubet

@unixman83: I'm aware of epoll. I was just curious, because there is mention of asynchronous I/O support on Linux, but I wasn't entirely sure whether there was support for sockets in particular (which BlackAura confirmed). As a side note, epoll may not be the best solution all the time (see sheddingbikes.com/posts/1280829388.html). – Peavy

Completion ports are not meant to multiplex sockets. They are perfectly useful with a single socket or file handle. They are a mechanism for data parallelism. Their purpose is to allow multiple threads to handle the data received asynchronously from a socket or a file. They are also used in thread pool implementations, such as .NET's. – Tacklind

Answer 1

So, on to my question... does linux support completion ports or even asynchronous I/O for sockets?

With regard to sockets, in 5.3 and later kernels, Linux has something analogous to completion ports in the shape of io_uring (for files/block devices io_uring support appeared in the 5.1 kernel).

Diamonddiamondback

Answer 2

If you're looking for something exactly like IOCP, you won't find it, because it doesn't exist.

Windows uses a notify on completion model (hence I/O Completion Ports). You start some operation asynchronously, and receive a notification when that operation has completed.

Linux applications (and most other Unix-alikes) generally use a notify on ready model. You receive a notification that the socket can be read from or written to without blocking. Then, you do the I/O operation, which will not block.

With this model, you don't need asynchronous I/O. The data is immediately copied into / out of the socket buffer.

The programming model for this is kind of tricky, which is why there are abstraction libraries like libevent. It provides a simpler programming model, and abstracts away the implementation differences between the supported operating systems.

There is a notify on ready model in Windows as well (select or WSAWaitForMultipleEvents), which you may have looked at before. It can't scale to large numbers of sockets, so it's not suitable for high-performance network applications.

Don't let that put you off - Windows and Linux are completely different operating systems. Something that doesn't scale well on one system may work very well on the other. This approach actually works very well on Linux, with performance comparable to IOCP on Windows.

Leija

Any synchronous I/O will block so long as you're writing more data than the buffer can hold. And if the data you're writing has been paged out, your process will block on the page fault. – Madelyn

@BlackAura: Wouldn't WSAAsyncSelect support a large number of sockets on Windows with the notify-on-ready model? Admittedly I've not used it with a large number of sockets, but since the socket handle is passed as an event parameter, and there's no bitmask or array of sockets to query, it seems like it ought to scale up magnificently. – Buckeye

@Ben Voigt The problem with WSAAsyncSelect is that it's built around the window procedure API. Its message loop is slow compared to the other polling functions. Having to handle thousands of sockets can significantly degrade the performance of your program. By the way, what do you by "there's no bitmask"? You still have to pass flags. – Peavy

@someguy: By "no bitmask" I mean no use of FD_SET, as is done in select. Also, WSAAsyncSelect requires using the thread message queue, not the window procedure API. It's perfectly possible to catch those messages in the message loop and not pass them through DispatchMessage and the window procedure code. – Buckeye

@Ben Voigt Oh, yeah, you're right about the messages being queued. However, you still have to poll using GetMessage. My point is that it's slow compared to other polling functions. I don't know too much about it, but I've been told that WSAAsyncSelect doesn't scale very well. – Peavy

@BlackAura, the models are not completely different. Asynchronous WSARecv() with 0 byte buffer is readiness notification, just like epoll_wait(). With high confidence (and depending on how big are the buffers you receive) the following WSARecv with > 0 size won't block, since the bytes are already cached by Winsock. – Profanatory

@Vladislav Vaintroub: WSARecv() returns 0 if the I/O operation has been scheduled; it doesn't mean the socket is ready for that operation. Also, any value other than 0 means there was an error. – Peavy

@someguy, GetQueuedCompletionStatus returns when WSARecv() with 0 bytes is completed, which then means "ready for operation". I understand it as - there is at least one byte on the connection that can be read without blocking – Profanatory

... or there is an error. GetQueuedCompletionStatus() is comparable to epoll_wait() , while asynchronous 0 bytes WSARecv() is comparable to epoll_ctl() with EPOLL_CTL_MOD|EPOLLIN|EPOLLET|EPOLLONESHOT flags – Profanatory

@Vladislav Vaintroub: WSARecv() schedules the I/O operation to be performed. When the operation is completed, GetQueuedCompletionStatus() will return. See the differences between readiness events and completion events. – Peavy

I do not think I ever denied it. You need to understand that WSARecv() 0 bytes operation is equivalent to "start watching for readiness". GetQueuedCompletionStatus in this case says "yes,something can be read from the socket". It is edge-triggered readiness notification, fully equivalent to epoll_ctl/epoll_wait example above. And epoll does not do completion notification – Profanatory

Answer 3

IOCP is pronounced "asynchronous I/O" on various UNIX platforms:

• POSIX AIO is the standard
• Kernel AIO, epoll and io_uring seem to be a Linux-specific implementations
• Kqueue is the *BSD and Mac OSX implementation
• Message Passing Interface (MPI) is an option for high-performance computing
• obligatory Boost reference - Boost.Asio

Coral

Thanks. Any chance you could provide me with some sample codes? I've looked into POSIX AIO before, but I couldn't find anything related to sockets. – Peavy

I haven't used POSIX AIO before. I just know that it has been discussed in various forums. Personally, I run BSD-based systems so I have used Kqueue for this. I would look into using Boost or some MPI implementation for concurrent I/O if that is what you are after. – Coral

I've looked into POSIX AIO and Kernel AIO, and both seem to imply that there is no true support for sockets, which is a shame. I'm going to assume that all the documents I've read are outdated, as Java 7 is apparently going to support asynchronous I/O, so linux has to have it, or will have it, right? Maybe I'll try and have a look at what Sun/Oracle have done so far. – Peavy

It does look like POSIX AIO has gotten short shrift. You might want to read some of references on "Fast UNIX Servers" (dank.qemfd.net/dankwiki/index.php/Fast_UNIX_Servers). There seems to be a lot of good information there. – Coral

"asynchronous I/O" is buggy and poorly supported on most Unices! Beware – Lubet

i was told completionPorts(IOCP) are much more faster than epoll/select, but linux now has io_uring witch is the linux version of IOCP. – Bevon

Answer 4

Use boost::asio. Hands down. It has a mild learning curve, but it's cross-platform, and automatically uses the best available method for the system you're compiling on. There's simply no reason not to.

I know that this isn't quite an answer to your question, but it's the best advice I could give.

Bleeding

+1 I hate ASIO because of its extreme use of namespaces, but I have to agree that it's the highest quality free library that fully supports BOTH Windows AND Linux. Libevent on the other hand has shoddy, unfinished support for Windows. – Lubet

Answer 5

Might be a bit late, but this answer is for the sake of completeness.

Linux, as of 2020 (v5.11), has introduced io_uring which is semantically similar to completion port on Windows but with very different APIs. In that, you queue an operation such as read() or write() using struct io_uring_sqe and get notifications via struct io_uring_cqe when it's done.

It also relies heavily on struct iovec, so familiarity with that structure and associated APIs is recommended. There is a tiny Linux-only library called liburing which makes access to the kernel APIs a lot easier.

Listed

The accepted answer points out io_uring, but thank you for the additional details. For anyone that doesn't mind using a high-level library, I believe libev supports io_uring. – Peavy

Answer 6

Read the blog entry from Google on libevent, you can implement IOCP semantics on Unix using asynchronous IO but cannot directly implement asynchronous IO semantics using IOCP,

http://google-opensource.blogspot.com/2010/01/libevent-20x-like-libevent-14x-only.html

For an example cross platform asynchronous IO with a BSD socket API look at ZeroMQ as recently published on LWN.net,

http://www.zeromq.org/

LWN article,

http://lwn.net/Articles/370307/

Revitalize

I don't think libevent supports asynchronous I/O, and it even says so on the linked page that it's for non-blocking sockets. I guess I'll look into it deeper later. 0MQ is also interesting, but I don't think it's what I'm looking for. It seems to be a message-passing library (similar to erlang actors?) – Peavy

@Peavy asynchronous I/O comprehends/includes non-blocking sockets. – Walhalla

Answer 7

Boost ASIO implements Windows style IOCP (Proactor design pattern) on Linux using epoll (Reactor pattern). See http://think-async.com/Asio/asio-1.5.3/doc/asio/overview/core/async.html

Discretion

Might be more useful to quote and summarize on that page from the linky supplied as link rot will happen and users will click on it to find 404 at some point. – Jeanninejeans