proc_open: Extending file descriptor numbers to enable "status" feedback from a Perl script

Question

PHP's proc_open manual states:

The file descriptor numbers are not limited to 0, 1 and 2 - you may specify any valid file descriptor number and it will be passed to the child process. This allows your script to interoperate with other scripts that run as "co-processes". In particular, this is useful for passing passphrases to programs like PGP, GPG and openssl in a more secure manner. It is also useful for reading status information provided by those programs on auxiliary file descriptors.

What Happens: I call a Perl script in a PHP-based web application and pass parameters in the call. I have no future need to send data to the script. Through stdout [1] I receive from the Perl script json_encoded data that I use in my PHP application.

What I would like to add: The Perl script is progressing through a website collecting information depending on the parameters passed in it's initial call. I would like to send back to the PHP application a text string that I could use to display as a sort of Progress Bar.

How I think I should do it: I would expect to poll (every 1-2 seconds) the channel that has been setup for that "progression" update. I would use Javascript / jQuery to write into an html div container for the user to view. I do not think I should mix the "progress" channel with the more critical "json_encode(data)" channel as I would then need to decipher the stdout stream. (Is this thought logical, practical?)

My Main Question: How do you use additional "file descriptors?" I would image the setup of additional channels to be straightforward, such as the 3 => ... in the below:

$tunnels = array(   
   0 => array('pipe', 'r'),     
   1 => array('pipe', 'w'),    
   2 => array('pipe', 'w'),    
   3 => array('pipe', 'w')        
);

$io = array();
$resource = proc_open("perl file/tomy/perl/code.pl $param1 $param2 $param3", $tunnels, $io);

if(!is_resource($resource)) {
    $error = "No Resource";
}

fclose($io[0]);

$perlOutput = stream_get_contents($io[1]);
$output = json_decode($perlOutput);

$errors = stream_get_contents($io[2]);
print "$errors<p>";

fclose($io[1]);
fclose($io[2]);

$result = proc_close($resource);

if($result != 0) {
    echo "you returned a $result result on proc_close";
}

But, in the Perl script I simply write to the stdout like:

my $json_terms = encode_json(\@terms);
print $json_terms;

If my understanding of setting up an additional channel is correct (above, the 3 => ...), then how would I write to it within the Perl script?

Thanks

Missy

Answer 1

Say you want to monitor the progress of a hello-world program, where each step is a dot written to the designated file descriptor.

#! /usr/bin/env perl

use warnings;
use strict;

die "Usage: $0 progress-fd\n" unless @ARGV == 1;

my $fd = shift;
open my $progress, ">&=", $fd or die "$0: dup $fd: $!";

# disable buffering on both handles
for ($progress, *STDOUT) {
  select $_;
  $| = 1;
}

my $output = "Hello, world!\n";

while ($output =~ s/^(.)(.*)\z/$2/s) {
  my $next = $1;
  print $next;
  print $progress ".";
  sleep 1;
}

Using bash syntax to open fd 3 on /tmp/progress and connect it to the program is

$ (exec 3>/tmp/progress; ./hello-world 3)
Hello, world!

$ cat /tmp/progress
..............

(It’s more amusing to watch live.)

To also see the dots on your terminal as they emerge, you could open your progress descriptor and effectively dup2 it onto the standard error—again using bash syntax and more fun in real time.

$ (exec 17>/dev/null; exec 17>&2; ./hello-world 17)
H.e.l.l.o.,. .w.o.r.l.d.!.
.

You could of course skip the extra step with

$ (exec 17>&2; ./hello-world 17)

to get the same effect.

If your Perl program dies with an error such as

$ ./hello-world 333
./hello-world: dup 333: Bad file descriptor at ./hello-world line 9.

then the write end of your pipe on the PHP side probably has its close-on-exec flag set.

Rodge

Clever. I'm not sure what the $ (exec 17>&2; ./hello-world 17) does. But now that I've got the channels between the php and perl script piped, I realize that I am stuck in an asynchronous ajax call to the php script which called the perl script. I guess I need a WebSocket(?). I'll mark either yours or @mob response soon. I appreciate the help from both of you. Thanks! – Missy

The first part makes fd 17 the same as fd 2 (which is the terminal's standard error), and then we tell hello-world to write progress indicators to fd 17. Because of how we set up the plumbing, the dots show up on the terminal too. mob is swell; give mob the checkmark. – Rodge

Answer 2

You open a new filehandle and dup it to file descriptor 3:

open STD3, '>&3';
print STDERR "foo\n";
print STD3   "bar\n";

$ perl script.pl 2> file2 3> file3
$ cat file2
foo
$ cat file3
bar

Edit: per Greg Bacon's comment, open STD3, '>&=', 3 or open STD3, '>&=3' opens the file descriptor directly, like C's fdopen call, avoiding a dup call and saving you a file descriptor.

Copland

More direct is open my $fh, ">&=", 3. – Rodge

mob, with yours and @Greg_Bacon help I've got the php <-> perl communication going on. Thanks. But even if I make the pipes hot I still can't write to a div in my web page until the ajax post - which calls the php which proc_opens the perl script - is done. I'm trying to give status updates on how the perl script is doing. Is long-polling or a webSocket my two best options? Thanks for your help! – Missy