Calling system commands from Perl

Question

In an older version of our code, we called out from Perl to do an LDAP search as follows:

# Pass the base DN in via the ldapsearch-specific environment variable 
# (rather than as the "-b" paramater) to avoid problems of shell 
# interpretation of special characters in the DN.
$ENV{LDAP_BASEDN} = $ldn;

$lcmd = "ldapsearch -x -T -1 -h $gLdapServer" .
        <snip>
        " > $lworkfile 2>&1";
system($lcmd);

if (($? != 0) || (! -e "$lworkfile"))
{
  # Handle the error
}

The code above would result in a successful LDAP search, and the output of that search would be in the file $lworkfile.

Unfortunately, we recently reconfigured openldap on this server so that a "BASE DC=" is specified in /etc/openldap/ldap.conf and /etc/ldap.conf. That change seems to mean ldapsearch ignores the LDAP_BASEDN environment variable, and so my ldapsearch fails.

I've tried a couple of different fixes but without success so far:

(1) I tried going back to using the "-b" argument to ldapsearch, but escaping the shell metacharacters. I started writing the escaping code:

my $ldn_escaped = $ldn;
$ldn_escaped =~ s/\/\\/g;
$ldn_escaped =~ s/`/\`/g;
$ldn_escaped =~ s/$/\$/g;
$ldn_escaped =~ s/"/\"/g;

That threw up some Perl errors because I haven't escaped those regexes properly in Perl (the line number matches the regex with the backticks in).

Backticks found where operator expected at /tmp/mycommand line 404, at end of line

At the same time I started to doubt this approach and looked for a better one.

(2) I then saw some Stackoverflow questions (here and here) that suggested a better solution.

Here's the code:

print("Processing...");

# Pass the arguments to ldapsearch by invoking open() with an array.
# This ensures the shell does NOT interpret shell metacharacters.
my(@cmd_args) = ("-x", "-T", "-1", "-h", "$gLdapPool",
                 "-b", "$ldn",
                 <snip>
                );

$lcmd = "ldapsearch";

open my $lldap_output, "-|", $lcmd, @cmd_args;

while (my $lline = <$lldap_output>)
{
  # I can parse the contents of my file fine
}

$lldap_output->close;

The two problems I am having with approach (2) are:

a) Calling open or system with an array of arguments does not let me pass > $lworkfile 2>&1 to the command, so I can't stop the ldapsearch output being sent to screen, which makes my output look ugly:

Processing...ldap_bind: Success (0)
        additional info: Success

b) I can't figure out how to choose which location (i.e. path and file name) to the file handle passed to open, i.e. I don't know where $lldap_output is. Can I move/rename it, or inspect it to find out where it is (or is it not actually saved to disk)?

Based on the problems with (2), this makes me think I should return back to approach (1), but I'm not quite sure how to

Bickerstaff

Answer 1

One approach would be to use IPC::Open3 to enable your Perl code to handle both the stdout and stderr streams of your external program.

Boaten

Answer 2

I would use IPC::Run3 for this. This is much like the open '-|' approach, but allows you to redirect STDERR too.

Note: $lldap_output is a pipe reading from ldapsearch. There's no file being created on disk.

If you want a file on disk, you could use IPC::Run3 like this:

use IPC::Run3;

my ($lcmd, @cmd_args) = ... # same as approach (2) above
my $lworkfile         = ... # same as approach (1) above

run3 [ $lcmd, @cmd_args ], undef, $lworkfile, $lworkfile;

This is like approach (1), but using -b instead of $ENV{LDAP_BASEDN}.

Kileykilgore

Is this in the core Perl modules? This solution looks good, but unfortunately Perl reports "Can't locate IPC/Run3.pm in @INC". I'm guessing I'd need to install this IPC-Run3-0.044 module, which I would prefer to avoid as I'd have to make build/install changes to our server. – Bickerstaff

No, IPC::Run3 is not a core module. – Kileykilgore

Answer 3

Thanks to Greg Hewgill for the answer. I'm posting my code below in case it helps anybody else wanting to use the open3 function.

use File::Copy;
use IPC::Open3;

# Pass the arguments to ldapsearch by invoking open() with an array.
# This ensures the shell does NOT interpret shell metacharacters.
my(@cmd_args) = ("-x", "-T", "-1", "-h", "$gLdapPool",
                 "-b", "$ldn",
                 <snip>
                );
$lcmd = "ldapsearch";
my $lldap_output;

# First arg is undef as I don't need to pass any extra input to the 
# process after it starts running.
my $pid = open3(undef, $lldap_output, $lldap_output, $lcmd, @cmd_args);

# Wait for the process to complete and then inspect the return code.
waitpid($pid, 0);

my $ldap_retcode = $? >> 8;

if ($ldap_retcode != 0)
{
  # Handle error
}

# Copy the output to $lworkfile so I can refer to it later if needed       
copy($lldap_output, $lworkfile);

while (my $lline = <$lldap_output>)
{
  # I can parse the contents of my file fine
}

$lldap_output->close;

Bickerstaff

Answer 4

See the docs for open. You can duplicate and redirect STDERR, run your command, then restore STDERR. It's more verbose than using any of the IPC::(Open3, Run, Run3, etc.) libraries, but possible to do without them if you can't/won't install extra modules, or don't want to use IPC::Open3.

Shayneshays

This would be great, but have you actually managed to do this? Command fails to redirect when I run: open DUP, ">&STDERR"; open DUP, ">", "duperr.txt"; open COMMAND, "-|", $my_command; open STDERR, ">DUP_STDERR"; – Middlemost

@schulwitz: You're restoring to the wrong filehandle AND doing it incorrectly, should be open STDERR, ">&DUP" – Shayneshays

Also, you duplicate stderr, but then open the duplicate to a file...you need to open STDERR to the file. – Shayneshays

Answer 5

Here's a hacky way to read both STDOUT and STDERR from an external program with multiple arguments using plain ol' "open":

my @command_with_arguments = (YOUR_PROGRAM, ARG1, ARG2, ARG3);
foreach(@command_with_arguments){s/'/'"'"'/g;}
foreach(@command_with_arguments){s/(.+)/'$1'/;}
my $run_command = join (' ', @command_with_arguments) . " 2>&1 |";
open my $program_output, $run_command;

Now just read $program_output to get both STDOUT and STDERR.

Middlemost