Linux: Executing child process with piped stdin/stdout

Question

Using Linux and C++, I would like a function that does the following:

string f(string s)
{
    string r = system("foo < s");
    return r;
}

Obviously the above doesn't work, but you get the idea. I have a string s that I would like to pass as the standard input of a child process execution of application "foo", and then I would like to record its standard output to string r and then return it.

What combination of Linux syscalls or POSIX functions should I use? I'm using Linux 3.0 and do not need the solution to work with older systems.

Cafeteria

possible duplicate of How can I run an external program from C and parse its output? – Juvenility

@Joe: Linux is an operating system. 3.0 is a version number. I'm indicating a platform specific answer that only works on Linux 3.0.0 and newer versions is acceptable. – Cafeteria

Then I suggest you look up the system calls pipe, fork, dup2 and exec. And maybe check a tutorial such as this one. – Juvenility

@user1131467: Sorry, it's just that system has become a recurring theme lately. For a quick list of reasons, it's a mix of security problems (shell quoting and other issues passing strings to system), incompatibility with multi-threaded programs, lack of any good way to process the command's output (your issue), and the fact that there's probably a much cleaner way to do what the external program would do, but to do it internal to your own process... – Stoffel

@R..: Thank you, but none of those issues apply to my case. I actually want what I am asking for, and for good reason. – Cafeteria

@user1131467: The suggestion by @Joachim is very valid. pipe, fork, and popen are intended for subproccess management. system is for fire and forget with simple commands such as date, or commands that do not require interaction such as apt-get (most of the time). – Rawson

If nothing else, you should do the equivalent of system but with somewhat saner signal handling... And probably also leave out the shell. – Stoffel

@R.. Note that OP did not actually ask for a solution using system. – Puri

(Commenting 5+ years later because this happened to show up on the front page.) Linux is a kernel. Release 3.0 of the Linux kernel was announced 2011-07-21. There are a plethora of operating systems that use the Linux kernel (and such systems are quite often referred to as "Linux"). Version numbers for such operating systems tend to be uncorrelated; you can't tell whether Fedora X is newer or older than Debian Y without looking up their release histories. My guess is that the OP was using version 3.0 of some unspecified distribution. We can't tell which, but it probably doesn't matter. – Larrylars

Answer 1

The code provided by eerpini does not work as written. Note, for example, that the pipe ends that are closed in the parent are used afterwards. Look at

close(wpipefd[1]); 

and the subsequent write to that closed descriptor. This is just transposition, but it shows this code has never been used. Below is a version that I have tested. Unfortunately, I changed the code style, so this was not accepted as an edit of eerpini's code.

The only structural change is that I only redirect the I/O in the child (note the dup2 calls are only in the child path.) This is very important, because otherwise the parent's I/O gets messed up. Thanks to eerpini for the initial answer, which I used in developing this one.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

#define PIPE_READ 0
#define PIPE_WRITE 1

int createChild(const char* szCommand, char* const aArguments[], char* const aEnvironment[], const char* szMessage) {
  int aStdinPipe[2];
  int aStdoutPipe[2];
  int nChild;
  char nChar;
  int nResult;

  if (pipe(aStdinPipe) < 0) {
    perror("allocating pipe for child input redirect");
    return -1;
  }
  if (pipe(aStdoutPipe) < 0) {
    close(aStdinPipe[PIPE_READ]);
    close(aStdinPipe[PIPE_WRITE]);
    perror("allocating pipe for child output redirect");
    return -1;
  }

  nChild = fork();
  if (0 == nChild) {
    // child continues here

    // redirect stdin
    if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1) {
      exit(errno);
    }

    // redirect stdout
    if (dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1) {
      exit(errno);
    }

    // redirect stderr
    if (dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1) {
      exit(errno);
    }

    // all these are for use by parent only
    close(aStdinPipe[PIPE_READ]);
    close(aStdinPipe[PIPE_WRITE]);
    close(aStdoutPipe[PIPE_READ]);
    close(aStdoutPipe[PIPE_WRITE]); 

    // run child process image
    // replace this with any exec* function find easier to use ("man exec")
    nResult = execve(szCommand, aArguments, aEnvironment);

    // if we get here at all, an error occurred, but we are in the child
    // process, so just exit
    exit(nResult);
  } else if (nChild > 0) {
    // parent continues here

    // close unused file descriptors, these are for child only
    close(aStdinPipe[PIPE_READ]);
    close(aStdoutPipe[PIPE_WRITE]); 

    // Include error check here
    if (NULL != szMessage) {
      write(aStdinPipe[PIPE_WRITE], szMessage, strlen(szMessage));
    }

    // Just a char by char read here, you can change it accordingly
    while (read(aStdoutPipe[PIPE_READ], &nChar, 1) == 1) {
      write(STDOUT_FILENO, &nChar, 1);
    }

    // done with these in this example program, you would normally keep these
    // open of course as long as you want to talk to the child
    close(aStdinPipe[PIPE_WRITE]);
    close(aStdoutPipe[PIPE_READ]);
  } else {
    // failed to create child
    close(aStdinPipe[PIPE_READ]);
    close(aStdinPipe[PIPE_WRITE]);
    close(aStdoutPipe[PIPE_READ]);
    close(aStdoutPipe[PIPE_WRITE]);
  }
  return nChild;
}

Middleaged

I'm trying to do something similar, but for me, this code causes, i believe, a deadlock for the following flow: parent: write, then read; child: stdin, then stdout. The problem goes away if i remove either one of the reads. The code is exactly what Ammo wrote. (It doesn't get stuck if the parent reads first and child writes) – Shaefer

Oh I understand your comment now. The example program does a blocking read(...) on the parent side, so it will in fact block forever if the child stops sending anything and does not close the stdout. This is outside the scope of this example program, which is just about getting the forking and redirects to be correct. You will need to write your own application logic to communicate between the parent and the child. – Middleaged

Here is a gist of a completely horrible application protocol (runs on MacOS, should work on Linux) that just sends a hello world message and assumes nothing goes wrong. You would of course write a real application protocol for production code, but at least you can see it running here. gist.github.com/derammo/e2802f9e4a713633901c7c5390388b78 – Middleaged

Answer 2

Since you want bidirectional access to the process, you would have to do what popen does behind the scenes explicitly with pipes. I am not sure if any of this will change in C++, but here is a pure C example :

void piped(char *str){
    int wpipefd[2];
    int rpipefd[2];
    int defout, defin;
    defout = dup(stdout);
    defin = dup (stdin);
    if(pipe(wpipefd) < 0){
            perror("Pipe");
            exit(EXIT_FAILURE);
    }
    if(pipe(rpipefd) < 0){
            perror("Pipe");
            exit(EXIT_FAILURE);
    }
    if(dup2(wpipefd[0], 0) == -1){
            perror("dup2");
            exit(EXIT_FAILURE);
    }
    if(dup2(rpipefd[1], 1) == -1){
            perror("dup2");
            exit(EXIT_FAILURE);
    }
    if(fork() == 0){
            close(defout);
            close(defin);
            close(wpipefd[0]);
            close(wpipefd[1]);
            close(rpipefd[0]);
            close(rpipefd[1]);
            //Call exec here. Use the exec* family of functions according to your need
    }
    else{
            if(dup2(defin, 0) == -1){
                    perror("dup2");
                    exit(EXIT_FAILURE);
            }
            if(dup2(defout, 1) == -1){
                    perror("dup2");
                    exit(EXIT_FAILURE);
            }
            close(defout);
            close(defin);
            close(wpipefd[1]);
            close(rpipefd[0]);
            //Include error check here
            write(wpipefd[1], str, strlen(str));
            //Just a char by char read here, you can change it accordingly
            while(read(rpipefd[0], &ch, 1) != -1){
                    write(stdout, &ch, 1);
            }
    }

}

Effectively you do this :

1. Create pipes and redirect the stdout and stdin to the ends of the two pipes (note that in linux, pipe() creates unidirectional pipes, so you need to use two pipes for your purpose).
2. Exec will now start a new process which has the ends of the pipes for stdin and stdout.
3. Close the unused descriptors, write the string to the pipe and then start reading whatever the process might dump to the other pipe.

dup() is used to create a duplicate entry in the file descriptor table. While dup2() changes what the descriptor points to.

Note : As mentioned by Ammo@ in his solution, what I provided above is more or less a template, it will not run if you just tried to execute the code since clearly there is a exec* (family of functions) missing, so the child will terminate almost immediately after the fork().

Barnard

Why are you retaining a reference to stdin/stdout with dup, when they are subsequently redirecting with dup2? dup2 implicitly closes its second parameter for you. – Gauze

You are right, that is unnecessary. Ammo's answer above is the right way of doing this, please refer to that. – Barnard

Obviously, you wanted to place the dup2() calls into the if(fork()==0) part before the close() lines and not into the else part. They are required in the if() part. – Arvell

Answer 3

Ammo's code has some error handling bugs. The child process is returning after dup failure instead of exiting. Perhaps the child dups can be replaced with:

    if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1 ||
        dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1 ||
        dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1
        ) 
    {
        exit(errno); 
    }

    // all these are for use by parent only
    close(aStdinPipe[PIPE_READ]);
    close(aStdinPipe[PIPE_WRITE]);
    close(aStdoutPipe[PIPE_READ]);
    close(aStdoutPipe[PIPE_WRITE]);

Malemute

You are correct. The child process should not be returning from that stack frame and also should not try to print errors. Probably nobody noticed because this never happens. – Middleaged

Replacing return with exit() is correct. I'd dispute that 'not try to print errors' is correct. No other process can report the error; standard error is for reporting errors. Very few programs should fail silently. – Gorilla