How to make pipes work with Runtime.exec()?

Question

Consider the following code:

String commandf = "ls /etc | grep release";

try {

    // Execute the command and wait for it to complete
    Process child = Runtime.getRuntime().exec(commandf);
    child.waitFor();

    // Print the first 16 bytes of its output
    InputStream i = child.getInputStream();
    byte[] b = new byte[16];
    i.read(b, 0, b.length); 
    System.out.println(new String(b));

} catch (IOException e) {
    e.printStackTrace();
    System.exit(-1);
}

The program's output is:

/etc:
adduser.co

When I run from the shell, of course, it works as expected:

poundifdef@parker:~/rabbit_test$ ls /etc | grep release
lsb-release

The internets tell me that, due to the fact that pipe behavior isn't cross-platform, the brilliant minds who work in the Java factory producing Java can't guarantee that pipes work.

How can I do this?

I am not going to do all of my parsing using Java constructs rather than grep and sed, because if I want to change the language, I'll be forced to re-write my parsing code in that language, which is totally a no-go.

How can I make Java do piping and redirection when calling shell commands?

Unbelief

I see it like this: If you do it with native Java string handling, you're guaranteed portability of the Java app to all the platforms Java support. OTOH, if you do it with shell commands, it's easier to change the langauge from Java, but will only work when you're on a POSIX platform. Few people change the app langauge rather than the platform the app runs on. That's why I think your reasoning is a bit curious. – Angleaangler

In the specific case of something simple like command | grep foo you are much better off with just running command and doing the filtering natively in Java. It makes your code somewhat more complex but you also reduce the overall resource consumption and attack surface significantly. – Blush

Answer 1

Write a script, and execute the script instead of separate commands.

Pipe is a part of the shell, so you can also do something like this:

String[] cmd = {
"/bin/sh",
"-c",
"ls /etc | grep release"
};

Process p = Runtime.getRuntime().exec(cmd);

Levasseur

@Levasseur What if you wanted to add options to ls i.e. ls -lrt ? – Cytoplast

@Levasseur I see that you are trying to use -c to specify a string of commands to the shell, but I don't understand why you have to make this a string array instead of just a single string? – Impulse

If someone is looking for android version here, then use /system/bin/sh instead – Sixteenmo

Answer 2

I ran into a similar problem in Linux, except it was "ps -ef | grep someprocess".
At least with "ls" you have a language-independent (albeit slower) Java replacement. Eg.:

File f = new File("C:\\");
String[] files = f.listFiles(new File("/home/tihamer"));
for (String file : files) {
    if (file.matches(.*some.*)) { System.out.println(file); }
}

With "ps", it's a bit harder, because Java doesn't seem to have an API for it.

I've heard that Sigar might be able to help us: https://support.hyperic.com/display/SIGAR/Home

The simplest solution, however, (as pointed out by Kaj) is to execute the piped command as a string array. Here is the full code:

try {
    String line;
    String[] cmd = { "/bin/sh", "-c", "ps -ef | grep export" };
    Process p = Runtime.getRuntime().exec(cmd);
    BufferedReader in =
            new BufferedReader(new InputStreamReader(p.getInputStream()));
    while ((line = in.readLine()) != null) {
        System.out.println(line); 
    }
    in.close();
} catch (Exception ex) {
    ex.printStackTrace();
}

As to why the String array works with pipe, while a single string does not... it's one of the mysteries of the universe (especially if you haven't read the source code). I suspect that it's because when exec is given a single string, it parses it first (in a way that we don't like). In contrast, when exec is given a string array, it simply passes it on to the operating system without parsing it.

Actually, if we take time out of busy day and look at the source code (at http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Runtime.java#Runtime.exec%28java.lang.String%2Cjava.lang.String[]%2Cjava.io.File%29), we find that is exactly what is happening:

public Process  [More ...] exec(String command, String[] envp, File dir) 
          throws IOException {
    if (command.length() == 0)
        throw new IllegalArgumentException("Empty command");
    StringTokenizer st = new StringTokenizer(command);
    String[] cmdarray = new String[st.countTokens()];
    for (int i = 0; st.hasMoreTokens(); i++)
        cmdarray[i] = st.nextToken();
    return exec(cmdarray, envp, dir);
}

Jenisejenkel

I'm seeing the same behavior with redirect, i.e. the '>' character. This extra analysis on String arrays is enlightening – Mares

You don't need to take a day out of your busy schedule - its written in front of your eyes docs/api/java/lang/Runtime.html#exec(java.lang.String) – Fermentative

Answer 3

Create a Runtime to run each of the process. Get the OutputStream from the first Runtime and copy it into the InputStream from the second one.

Stonecrop

It's worth pointing out that this is far less efficient than an os-native pipe, since you are copying memory into the Java process's address space and then back out to the subsequent process. – Zibet

Answer 4

@Kaj accepted answer is for linux. This is the equivalent one for Windows:

String[] cmd = {
"cmd",
"/C",
"dir /B | findstr /R /C:"release""
};
Process p = Runtime.getRuntime().exec(cmd);

Putman

Don't you have to use \" – Inquiring