Reading from STDIN pipe when using proc_open

Question

I am trying to make a website where people can compile and run their code online, thus we need to find an interactive way for users to send instructions.

Actually, what first comes to mind is exec() or system(), but when users want to input sth, this way won't work. So we have to use proc_open().

For instance, the following code

int main()
{
    int a;
    printf("please input a integer\n");
    scanf("%d", &a);
    printf("Hello World %d!\n", a);
    return 0;
}

When I used proc_open(), like this

$descriptorspec = array(      
0 => array( 'pipe' , 'r' ) ,  
    1 => array( 'pipe' , 'w' ) ,  
    2 => array( 'file' , 'errors' , 'w' ) 
);  
$run_string = "cd ".$addr_base."; ./a.out 2>&1";
$process = proc_open($run_string, $descriptorspec, $pipes);
if (is_resource($process)) {
    //echo fgets($pipes[1])."<br/>";
    fwrite($pipes[0], '12');
    fclose($pipes[0]);
    while (!feof($pipes[1]))
        echo fgets($pipes[1])."<br/>";
    fclose($pipes[1]);
    proc_close($process);
}

When running the C code, I want to get the first STDOUT stream, and input the number, then get the second STDOUT stream. But if I have the commented line uncommented, the page will be blocked.

Is there a way to solve the problem? How can I read from the pipe while not all data has been put there? Or is there a better way to write this kind of interactive program?

Range

Is your "user" interacting from website? Because by that way user does not seem to have direct access of your server's STDIN. – Lineolate

@Lineolate The user presses a button to compile, and inputs x and sends it to the server. But before he inputs x, the server has to get the stream from STDIN first and sends it to the website so that the user knows he should input x. The problem is the server cannot get the stream at that moment.. – Range

Answer 1

It is more a C or a glibc problem. You'll have to use fflush(stdout).

Why? And what's the difference between running a.out in a terminal and calling it from PHP?

Answer: If you run a.out in a terminal (being stdin a tty) then the glibc will use line buffered IO. But if you run it from another program (PHP in this case) and it's stdin is a pipe (or whatever but not a tty) than the glibc will use internal IO buffering. That's why the first fgets() blocks if uncommented. For more info check this article.

Good news: You can control this buffering using the stdbuf command. Change $run_string to:

$run_string = "cd ".$addr_base.";stdbuf -o0 ./a.out 2>&1";

---

Here comes a working example. Working even if the C code don't cares about fflush() as it is using the stdbuf command:

Starting subprocess

$cmd = 'stdbuf -o0 ./a.out 2>&1';

// what pipes should be used for STDIN, STDOUT and STDERR of the child
$descriptorspec = array (
    0 => array("pipe", "r"),
    1 => array("pipe", "w"),
    2 => array("pipe", "w")
 );

// open the child
$proc = proc_open (
    $cmd, $descriptorspec, $pipes, getcwd()
);

set all streams to non blocking mode

// set all streams to non blockin mode
stream_set_blocking($pipes[1], 0);
stream_set_blocking($pipes[2], 0);
stream_set_blocking(STDIN, 0);

// check if opening has succeed
if($proc === FALSE){
    throw new Exception('Cannot execute child process');
}

get child pid. we need it later

// get PID via get_status call
$status = proc_get_status($proc);
if($status === FALSE) {
    throw new Exception (sprintf(
        'Failed to obtain status information '
    ));
}
$pid = $status['pid'];

poll until child terminates

// now, poll for childs termination
while(true) {
    // detect if the child has terminated - the php way
    $status = proc_get_status($proc);
    // check retval
    if($status === FALSE) {
        throw new Exception ("Failed to obtain status information for $pid");
    }
    if($status['running'] === FALSE) {
        $exitcode = $status['exitcode'];
        $pid = -1;
        echo "child exited with code: $exitcode\n";
        exit($exitcode);
    }

    // read from childs stdout and stderr
    // avoid *forever* blocking through using a time out (50000usec)
    foreach(array(1, 2) as $desc) {
        // check stdout for data
        $read = array($pipes[$desc]);
        $write = NULL;
        $except = NULL;
        $tv = 0;
        $utv = 50000;

        $n = stream_select($read, $write, $except, $tv, $utv);
        if($n > 0) {
            do {
                $data = fread($pipes[$desc], 8092);
                fwrite(STDOUT, $data);
            } while (strlen($data) > 0);
        }
    }


    $read = array(STDIN);
    $n = stream_select($read, $write, $except, $tv, $utv);
    if($n > 0) {
        $input = fread(STDIN, 8092);
        // inpput to program
        fwrite($pipes[0], $input);
    }
}

Azral

Thanks~ But the C code is from the user, and he just presses Enter to compile and run. Is it possible to change some part of test.php to make it work? – Range

Interesting problem! :) Will investigate this. – Azral

I wonder if this works on Mac OSX? It shows sh: stdbuf: command not found. I will try on Ubuntu later! – Range

thx a lot, it worked! but how can I tell whether everything in pipes[1] has been read? when using while(!feof($pipes[1])), it blocks again.. – Range

Sorry, didn't saw your comment before. Use non-blocking streams, see stream_set_blocking(). Also you'll have to use proc_get_status() and check for running state. – Azral

I used these codes, stream_set_blocking($pipes[1], 0);echo fgets($pipes[1]);, it turned out nothing came out at all.. I don't know what went wrong.. – Range

have you tested my example? – Azral

Check my example. I'm using blocking sockets but stream_select() with a timeout of 50000 usec. Try my example (have you realized that I gave a second example?) – Azral

can you send me full version of your example? Some errors just came out, so I edited some places but it blocked.. [email protected] – Range

Its from a more complex class.. Just remove the $this and add the vars $cmd, $descriptorspec, $pipes, $cwd, $env,$otherOptions... (You already have them) – Azral

I did as you said.. and I run the example in terminal but it seems to have no response to what I input.. – Range

Check my update. I have tested it and it worked ( With your unchanged C code) – Azral

Thats funny :) It works even with the $cmd = 'stdbuf -o0 /bin/bash' but without prompt (of course). Never tested it before. – Azral

This time it worked! Appreciate your help. I will try it on my project! – Range

Holy guacamole. I have had this problem for a year now and this is the only working solution. I will dig into the code and try to better my understanding of sockets, pipes and all the like. – Bridgeboard

Just a quick update, I am using stdbuf -o0 with symfony/process (instead of the code above), and it works without any issues. – Bridgeboard

@AvindraGoolcharan Yes, if you just call a single command stdbuf -o0 is fine. The code above is meant to send multiple commands into the command's stdin. – Azral

@hek2mgl: I missed that point about stdin. But I was also using this snippet to do line by line processing, which symfony/process does. You can asynchronously process output line by line with a callback. – Bridgeboard

Answer 2

The answer is surprisingly simple: leave $descriptorspec empty. If you do so, the child process will simply use the STDIN/STDOUT/STDERR streams of the parent.

➜  ~  ✗ cat stdout_is_atty.php
<?php

var_dump(stream_isatty(STDOUT));
➜  ~  ✗ php -r 'proc_close(proc_open("php stdout_is_atty.php", [], $pipes));'
/home/chx/stdout_is_atty.php:3:
bool(true)
➜  ~  ✗ php -r 'passthru("php stdout_is_atty.php");'
/home/chx/stdout_is_atty.php:3:
bool(false)
➜  ~  ✗ php -r 'exec("php stdout_is_atty.php", $output); print_r($output);'
Array
(
    [0] => /home/chx/stdout_is_atty.php:3:
    [1] => bool(false)
)

Credit goes to John Stevenson, one of the maintainers of composer.

If you are interested why this happens: PHP does nothing for empty descriptors and uses the C / OS defaults which just happens to be the desired one.

So the C code responsible for proc_open always merely iterates the descriptors. If there are no descriptors specified then all that code does nothing. After that, the actual execution of the child -- at least on POSIX systems -- happens via calling fork(2) which makes the child inherit file descriptors (see this answer). And then the child calls one of execvp(3) / execle(3) / execl(3) . And as the manual says

The exec() family of functions replaces the current process image with a new process image.

Perhaps it's more understandable to say the memory region containing the parent is replaced by the new program. This is accessible as /proc/$pid/mem, see this answer for more. However, the system keeps a tally of the opened files outside of this region. You can see them in /proc/$pid/fd/ -- and STDIN/STDOUT/STDERR are just shorthands for file descriptors 0/1/2. So when the child replaces the memory, the file descriptors just stay in place.

Pacificas