Using poll function with buffered streams

Question

I am trying to implement a client-server type of communication system using the poll function in C. The flow is as follows:

1. Main program forks a sub-process
2. Child process calls the exec function to execute some_binary
3. Parent and child send messages to each other alternately, each message that is sent depends on the last message received.

I tried to implement this using poll, but ran into problems because the child process buffers its output, causing my poll calls to timeout. Here's my code:

int main() {
char *buffer = (char *) malloc(1000);
int n;

pid_t pid; /* pid of child process */

int rpipe[2]; /* pipe used to read from child process */
int wpipe[2]; /* pipe used to write to child process */
pipe(rpipe);
pipe(wpipe);

pid = fork();
if (pid == (pid_t) 0)
{
    /* child */

    dup2(wpipe[0], STDIN_FILENO);
    dup2(rpipe[1], STDOUT_FILENO);
    close(wpipe[0]); close(rpipe[0]);
    close(wpipe[1]); close(rpipe[1]);
    if (execl("./server", "./server", (char *) NULL) == -1)
    {
        fprintf(stderr, "exec failed\n");
        return EXIT_FAILURE;
    }       
    return EXIT_SUCCESS;
}
else
{
    /* parent */

    /* close the other ends */
    close(wpipe[0]);
    close(rpipe[1]);

    /* 
      poll to check if write is good to go 
                This poll succeeds, write goes through
        */
    struct pollfd pfds[1];
    pfds[0].fd = wpipe[1];
    pfds[0].events = POLLIN | POLLOUT;
    int pres = poll(pfds, (nfds_t) 1, 1000);
    if (pres > 0)
    {
        if (pfds[0].revents & POLLOUT)
        {
            printf("Writing data...\n");
            write(wpipe[1], "hello\n", 6);
        }
    }

    /* 
        poll to check if there's something to read.
        This poll times out because the child buffers its stdout stream.
    */
    pfds[0].fd = rpipe[0];
    pfds[0].events = POLLIN | POLLOUT;
    pres = poll(pfds, (nfds_t) 1, 1000);
    if (pres > 0)
    {
        if (pfds[0].revents & POLLIN)
        {
            printf("Reading data...\n");
            int n = read(rpipe[0], buffer, 1000);
            buffer[n] = '\0';
            printf("child says:\n%s\n", buffer);
        }
    }

    kill(pid, SIGTERM);
    return EXIT_SUCCESS;
}
}

The server code is simply:

int main() {
    char *buffer = (char *) malloc(1000);

    while (scanf("%s", buffer) != EOF)
    {
        printf("I received %s\n", buffer);
    }   
    return 0;
}

How do I prevent poll calls from timing out because of buffering?

EDIT:

I would like the program to work even when the execed binary is external, i.e., I have no control over the code - like a unix command, e.g., cat or ls.

Dorm

why the child process closes all pipes? ` close(wpipe[0]); close(rpipe[0]); close(wpipe[1]); close(rpipe[1]);` ???? – Roustabout

@GiuseppePes: The fds are duplicated as stdin/stdout, so closing them seems to be OK. – Cruciform

@MartinR thanks! I missed the dub2 calls! :( – Roustabout

You should have an event loop so the poll should be inside a loop.... – Chronaxie

The server code should call fflush inside the while(scanf loop... – Chronaxie

Answer 1

You need, as I answered in a related answer to a previous question by you, to implement an event loop; as it name implies, it is looping, so you should code in the parent process:

while (1) { // simplistic event loop!
   int status=0;
   if (waitpid(pid, &status, WNOHANG) == pid)
      { // clean up, child process has ended
        handle_process_end(status);
        break;
      };
   struct pollpfd pfd[2];
   memset (&pfd, 0, sizeof(pfd)); // probably useless but dont harm
   pfd[0].fd = rpipe[0];
   pfd[0].events = POLL_IN;
   pfd[1].fd = wpipe[1];
   pfd[0].event = POLL_OUT;
   #define DELAY 5000 /* 5 seconds */
   if (poll(pfd, 2, DELAY)>0) {
      if (pfd[0].revents & POLL_IN) {
         /* read something from rpipe[0]; detect end of file; 
            you probably need to do some buffering, because you may 
            e.g. read some partial line chunk written by the child, and 
            you could only handle full lines. */
      };
      if (pfd[1].revents & POLL_OUT) {
         /* write something on wpipe[1] */
      };
   }
   fflush(NULL);
} /* end while(1) */

you cannot predict in which order the pipes are readable or writable, and this can happen many times. Of course, a lot of buffering (in the parent process) is involved, I leave the details to you.... You have no influence on the buffering in the child process (some programs detect that their output is or not a terminal with isatty).

What an event polling loop like above gives you is to avoid the deadlock situation where the child process is blocked because its stdout pipe is full, while the parent is blocked writing (to the child's stdin pipe) because the pipe is full: with an event loop, you read as soon as some data is polled readable on the input pipe (i.e. the stdout of the child process), and you write some data as soon as the output pipe is polled writable (i.e. is not full). You cannot predict in advance in which order these events "output of child is readable by parent" and "input of child is writable by parent" happen.

I recommend reading Advanced Linux Programming which has several chapters explaining these issues!

BTW my simplistic event loop is a bit wrong: if the child process terminated and some data remains in its stdout pipe, its reading is not done. You could move the waitpid test after the poll

Also, don't expect that a single write (from the child process) into a pipe would trigger one single read in the parent process. In other words, there is no notion of message length. However, POSIX knows about PIPE_MAX .... See its write documentation. Probably your buffer passed to read and write should be of PIPE_MAX size.

I repeat: you need to call poll inside your event loop and very probably poll will be called several times (because your loop will be repeated many times!), and will report readable or writable pipe ends in an unpredictable (and non-reproducible) order! A first run of your program could report "rpipe[0] readable", you read 324 bytes from it, you repeat the event loop, poll says you "wpipe[1] writable", you can write 10 bytes to it, you repeat the event loop, poll tells that "rpipe[0] readable", you read 110 bytes from it, you repeat the event loop, poll tells again "rpipe[0] readable", you read 4096 bytes from it, etc etc etc... A second run of the same program in the same environment would give different events, like: poll says that "wpipe[1] writable", you write 1000 bytes to it, you repeat the loop, poll says that "rpipe[0] readable, etc....

NB: your issue is not the buffering in the child ("client") program, which we assume you cannot change. So what matters is not the buffered data in it, but the genuine input and output (that is the only thing your parent process can observe; internal child buffering is irrelevant for the parent), i.e. the data that your child program has been able to really read(2) and write(2). And if going thru a pipe(7), such data will become poll(2)-able in the parent process (and your parent process can read or write some of it after POLL_IN or POLL_OUT in the updated revents field after poll). BTW, if you did code the child, don't forget to call fflush at appropriate places inside it.

Chronaxie

Consider the following sequence of events: 1. parent writes on wpipe[1] 2. child reads from stdin (to which I have copied wpipe[0]), outputs its response onto stdout 3. Since stdout in the child process is buffered, the response does not reach parent immediately 4. parent cannot send its next message, because, as I stated in the question, the response is determined after processing the last received message. parent has not received any message from the child process so far. Hence, the parent program blocks. How can this be handled with a simple event loop? – Dorm

You should not care about what is buffered in the child process. You cannot change that. Just care about what is really written and read by the child, that is about what is respectively readable and writable by the parent, hence the event loop which should be really repeated, like every loop.... – Chronaxie

The important thing is that you need a loop and you need to call poll many times inside such a loop.... – Chronaxie

Answer 2

There seem to be two problems in your code. "stdout" is by default buffered, so the server should flush it explicitly:

printf("I received %s\n", buffer);
fflush(stdout);

And the main program should not register for POLLOUT when trying to read (but you may want register for POLLERR):

pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLERR;

With these modifications you get the expected output:

$ ./main
Writing data...
Reading data...
child says:
I received hello

Generally, you should also check the return value of poll(), and repeat the call if necessary (e.g. in the case of an interrupted system call or timeout).

Cruciform

The above solution will work if I can modify the server code, but what if I can't? What if I want to run an external program like the unix command cat? – Dorm

@prvnsmpth: From your question I had the impression that you own the server code and therefore can modify it. And even with "cat" it seems to work if you set pfds[0].events = POLLIN | POLLERR like I suggested. But if the external program buffers it output (and therefore does not write to stdout), then I do not think there is anything that you can do. – Cruciform

@prvnsmpth: If the external process reads the input, but does not write the response, there is nothing you can do. – Cruciform

poll should always be repeated inside a loop because a pipe could be readable many times!!!! (not only because of interrupted syscalls or timeouts) – Chronaxie

@BasileStarynkevitch: You are right of course, and I forgot to mention that reason when I indicated that poll should be repeated if necessary. - However I have the feeling (in particular after the question has been edited) that there is no solution to OP's problem. If the server does not respond then there is nothing that the client can do. – Cruciform