epoll_wait always sets EPOLLOUT bit?

Question

On a listening socket I set the EPOLLIN bit however on client connections I set EPOLLIN | EPOLLOUT bits to struct epoll_event like so:

struct epoll_event ev;

ev.data.fd = fd;
ev.events = EPOLLIN | EPOLLOUT;
if (epoll_ctl(evs->epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0)
     ...

And this is how I test bits:

if ((events & EPOLLIN) == EPOLLIN)
     ...
if ((events & EPOLLOUT) == EPOLLOUT)
     ...

I've also tried like:

if (events & EPOLLIN)
     ...
if (events & EPOLLOUT)
     ...

Both ways are ALWAYS true!

However, whenever I call epoll_wait on my epoll fd, the active file descriptor returned ALWAYS has both bits set even though send() didn't return EAGAIN but when I try to recv() it returns EAGAIN.

I have no idea what am I supposed to do when recv() returns EAGAIN, am I supposed to remove the EPOLLOUT flag or what?

More code as requested by @Nikolai N Fetissov:

static int get_active_fd(events *evs, int index, sstate_t *flags)
{
    uint32_t events = evs->events[index].events;
    int fd = evs->events[index].data.fd;;

    if ((events & EPOLLERR) == EPOLLERR || (events & EPOLLHUP) == EPOLLHUP) {
        close(fd);
        return -1;
    }

    if (events & EPOLLIN)
        *flags |= DATA_IN;

    return fd;
}

void sockset_add(events *evs, int fd)
{
    struct epoll_event ev;
    ...
    ev.data.fd = fd;
    ev.events = EPOLLIN;
    if (epoll_ctl(evs->epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0)
        eprintf("sockset_add(): epoll_ctl(%d) returned an error %d(%s)\n",
                fd, errno, strerror(errno));
}

Then later where I call epoll_wait():

if (flags & DATA_IN) {
       /* try to read which is impossible because this is never set.  */

Merrill

Answer 1

I think you got it backwards. Don't include EPOLLOUT unless you got EAGAIN from a write attempt, and remove it when you have successfully written bytes to a socket.

This basically means that socket is always writable as long as there's space in socket in-kernel send buffer.

Edit 0:

You have two data streams - input and output.

You are waiting for the input by including EPOLLIN in the flags. If upon return from epoll_wait(2) that flag is not set, then either some event happened on some other socket, or this socket had some other event. Leave the flag in the events unless you get an error (meaning you are still interested in the input on the socket).

You don't have to wait for the output (since it's your action), you just write to the socket, but if you overflow socket send buffer, you'll get EAGAIN from send(2) or write(2). In this case you start waiting for output to be possible (kernel draining socket send buffer thus making room for you to send more) by including EPOLLOUT. Once you get that, write your pending output bytes to the socket, and if you are successful, remove EPOLLOUT from the events.

Now EPOLLET indicates edge-triggered wait, meaning your desired event would only be signaled once per state change (like from "no input" to "there's input"). In this mode you are supposed to read input bytes in a loop until you get EAGAIN.

Hope this helps.

Photochronograph

I removed the bit but still testing the bit results to true – Merrill

I mean, if I remove EPOLLOUT and test for EPOLLIN then it would fail, which is utterly confusing – Merrill

That's what I' saying - you are confused, so show how you setup descriptors for polling and how you test poll results. The mixup is somewhere there. – Photochronograph

Okay, let me guess, do I need EPOLLET to replace EPOLLOUT when there's no need to write data (aka when send() doesn't return EAGAIN)? EDIT: More code added now. – Merrill

@user9000 If you want to read data, set EPOLLIN , and read data when it returns true, if reading returns EAGAIN, wait till next time. If you have data to send, send it. If sending returns EAGAIN and/or EWOULDBLOCK, you turn on EPOLLOUT and resume sending when EPOLLOUT becomes true. If you do not have anything to send you turn off EPOLLOUT. It's hard from your code to see which bits you're checking in which structs though, you should probably show the code that does epoll_wait(). EPOLLET is hard to get right, I'd advice you to not use it until you understand epoll absolutely 100% completely. – Postfix