Why having to use non-blocking fd in a edge triggered epoll function?

Question

I read document abount edge triggered epoll function in web as follows:

1. The file descriptor that represents the read side of a pipe (rfd) is registered on the epoll instance.
2. A pipe writer writes 2 kB of data on the write side of the pipe.
3. A call to epoll_wait(2) is done that will return rfd as a ready file descriptor.
4. The pipe reader reads 1 kB of data from rfd.
5. A call to epoll_wait(2) is done.
.......
.......

The suggested way to use epoll as an edge-triggered (EPOLLET) interface is as follows: i) Use nonblocking file descriptors ii) Call epoll_wait for an event only after read(2) or write(2) return EAGAIN.

I understood 2, but I couldn't know why nonblocking file descriptors are used.

Could anyone explain the reason why nonblocking file descriptors are used? Why is it all right to use blocking file descriptors in a level triggered epoll function?

Birddog

Answer 1

The idea is to try to completely drain the file descriptor when you have an edge-triggered notification that there is data to be had. So, once epoll() returns, you loop over the read() or write() until it returns -EAGAIN when there is no more data to be had.

If the fd was opened blocking, then this last read() or write() would also block, and you wouldn't have the chance to go back to the epoll() call to wait on the entire set of fds. When opened nonblocking, the last read()/write() does return, and you do have the chance to go back to polling.

This is not so much of a concern when using epoll() in a level-triggered fashion, since in this case epoll() will return immediately if there is any data to be had. So a (pseudocode) loop such as:

while (1) {
  epoll();
  do_read_write();
}

would work, as you're guaranteed to call do_read_write() as long as there is data. When using edge-triggered epoll, there is potential for the notification that new data is available to be missed, if it comes in between the finish of do_read_write() and the next call to epoll().

Spit

How to deal with the "potential for the notification that new data is available to be missed"? – Skipton

The last portion of this answer doesn't seem to be on point. The events that occurred while you are not waiting will be registered as well and returned next time epoll_wait is run. But the real problem is if you don't read all the data from the socket by that do_read_write function and you get to epoll_wait again. Then with edge-triggered behavior, you will be stuck there indefinitely because then the events are only produced if there is a change in readability (or writeability). The problem is not when the data come but that you didn't read them all. – Keen

Answer 2

I guess it's because of the semantics of edge-triggered. The edge-trigger, according to the semantics, will raise another event only once the EAGAIN has been received. In case of blocking sockets there is no EAGAIN. You could have defined it in some other way, but this how Linux defines it. In other words, If you use blocking sockets, you have no idea about when you can safely call epoll_wait.

Darky

Answer 3

You must read all or write all the data on epoll's ET mode, because the et mode triggered once after the flag changed. When you have read all data, the thread must be hang if you use the block read or write. So that nonblocking must be used.

Montparnasse