Read unbuffered data from pipe in Perl

Question

I am trying to read unbufferd data from a pipe in Perl. For example in the program below:

open FILE,"-|","iostat -dx 10 5";
$old=select FILE;
$|=1;
select $old;
$|=1;

foreach $i (<FILE>) {
  print "GOT: $i\n";
}

iostat spits out data every 10 seconds (five times). You would expect this program to do the same. However, instead it appears to hang for 50 seconds (i.e. 10x5), after which it spits out all the data.

How can I get the to return whatever data is available (in an unbuffered manner), without waiting all the way for EOF?

P.S. I have seen numerous references to this under Windows - I am doing this under Linux.

Willianwillie

You should be using while not a foreach. And your output buffering is immaterial, given that it’s an input handle not an output handle. – Matthei

What is the difference when using while and foreach ? – Lovelovebird

@alertjean: In foreach my $i (<FILE>) { ... }, the file read is done in list context, that is, the whole file is read before the lines are processed in the foreach loop. In while (my $i = <FILE>) { ... }, the read is done in scalar context, that is, each line is read and then processed in the while block, before the next line is read. – Longsighted

Answer 1

#!/usr/bin/env perl

use strict;
use warnings;



open(PIPE, "iostat -dx 10 1 |")       || die "couldn't start pipe: $!";

while (my $line = <PIPE>) {
    print "Got line number $. from pipe: $line";
}

close(PIPE)                           || die "couldn't close pipe: $! $?";

Matthei

Answer 2

If it is fine to wait in your Perl script instead on the linux command, this should work. I don't think Linux will give control back to the Perl script before the command execution is completed.

#!/usr/bin/perl -w
my $j=0;
while($j!=5)
{
    open FILE,"-|","iostat -dx 10 1";
    $old=select FILE;
    $|=1;
    select $old;
    $|=1;

    foreach $i (<FILE>)
    {
        print "GOT: $i";
    }
    $j++;
    sleep(5);
}

Lovelovebird

Setting $| only affects output buffering. You cannot affect the other program’s output buffering that way. Plus what’s with the weird open instead of just open(PIPEHANDLE, "iostat -dx 10 1 |") || die "can't start pipe";? I can never remember "|-" vs "-|" myself. Anyway, if the iostat program isn’t flushing its buffers over a pipe, you have to turn to ptys, which is a real bother but unavoiable. – Matthei

Answer 3

I have below code working for me

#!/usr/bin/perl
use strict;
use warnings;
open FILE,"-|","iostat -dx 10 5";

while (my $old=<FILE>)
{
  print "GOT: $old\n";
}

Goodill

Why not open(FILE, "iostat -dx 10 5 |") ? – Matthei

@tchist because open PIPE, '-|', 'iostat', '-dx', 10, 5 or die $! – Ogbomosho

Answer 4

The solutions so far did not work for me with regards to unbuffering (Windows ActiveState Perl 5.10).

According to http://perldoc.perl.org/PerlIO.html, "To get an unbuffered stream specify an unbuffered layer (e.g. :unix ) in the open call:".

So

open(PIPE, '-|:unix', 'iostat -dx 10 1') or die "couldn't start pipe: $!";

while (my $line = <PIPE>) {
    print "Got $line";
}

close(PIPE);

which worked in my case.

Thermoelectricity