How to redirect ProcessBuilder's output to a string?

Question

I am using the following code to start a process builder. I want to know how can I redirect its output to a String.

ProcessBuilder pb = new ProcessBuilder(
    System.getProperty("user.dir") + "/src/generate_list.sh", filename);
Process p = pb.start();

I tried using ByteArrayOutputStream but it didn't seem to work.

Goodlooking

how you used the ByteArrayOutputStream? – Polyandrous

'It didn't seem to work' is not a problem description. ProcessBuilder doesn't have streams, but Process does. You aren't starting a ProcessBuilder, you are using it to create a Process, and then starting that. Be precise. – Scram

Answer 1

Read from the InputStream. You can append the output to a StringBuilder:

BufferedReader reader = 
                new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder builder = new StringBuilder();
String line = null;
while ( (line = reader.readLine()) != null) {
   builder.append(line);
   builder.append(System.getProperty("line.separator"));
}
String result = builder.toString();

Mascon

You might want to add how to get the error stream, or both. – Galibi

@Galibi This can easily be done using processBuilder.inheritIO() – Mascon

@Mascon this discards the original end-of-line character(s) and replaces with that of the current platform, which could differ. – Neath

@Mascon even though I get the result correct my line is null, why is that? – Janetjaneta

Note that if you do not redirect the error stream, you will need to capture the outputs from the error stream as well as the input stream, otherwise the process will hang. You have to collect the streams in separate threads otherwise your main thread will hang. In this case, search for "gobbler thread" for the full solution. – Rus

If you want stdout+stderr as one you can use ProcessBuilder.redirectErrorStream(boolean) – Exude

You can improve it with Java Stream: BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream())); String stringBuilder = reader.lines().collect(Collectors.joining(System.lineSeparator())); – Honeysucker

Is there any way to combine this solution and waiting for the process with a timeout? Normally you wait with a timeout using Process#waitFor. But I think this solution will block until the process exits, at which point it will be to late to use waitFor. – Aeronautics

Answer 2

As of Java 9, we finally have a one liner:

ProcessBuilder pb = new ProcessBuilder("pwd");
Process process = pb.start();

String result = new String(process.getInputStream().readAllBytes());

Engelbert

Is there any way to combine this solution and waiting for the process with a timeout? Normally you wait with a timeout using Process#waitFor. But I think readAllBytes will block until the process exits, at which point it will be to late to use waitFor. – Aeronautics

I wish there was a way to simply pass an outputStream and let the jvm write to the stream instead of manually reading from the process. It makes processing better. – Racklin

This solution is wrong as you a.) need to read both stdout AND stdin and b.) do this on separate threads. The reason (as already explained by Richard Whitehead above) is that the process will write stdout/stderr output to fixed-size buffers and if those get full and are never read, the process will block indefinitely. – Matey

Answer 3

Using Apache Commons IOUtils you can do it in one line:

ProcessBuilder pb = new ProcessBuilder("pwd");
String output = IOUtils.toString(pb.start().getInputStream(), StandardCharsets.UTF_8);

Tuinal

The stream still needs to be closed afterward, right? – Dowry

Not sure. I haven't seen any ProcessBuilder examples that close the InputStream. An answer about Process says that you should still close the stream.... #7098197 – Tuinal

This method is deprecated in newer versions of IOUtils, providing the encoding is now the preferred usage. eg. String output = IOUtils.toString(pb.start().getInputStream(), "UTF-8"); – Prism

process.waitFor() hangs on yt-dlp --ignore-errors --dump-json http://music.youtube.com/watch?v=SAsJ_n747T4 http://music.youtube.com/watch?v=oDLU3e34G1o hence I can't read the output. I can see the output though by using processBuilder.inheritIO(). – Eun

Answer 4

Java 8 example:

public static String runCommandForOutput(List<String> params) {
    ProcessBuilder pb = new ProcessBuilder(params);
    Process p;
    String result = "";
    try {
        p = pb.start();
        final BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));

        StringJoiner sj = new StringJoiner(System.getProperty("line.separator"));
        reader.lines().iterator().forEachRemaining(sj::add);
        result = sj.toString();

        p.waitFor();
        p.destroy();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return result;
}

Usage:

List<String> params = Arrays.asList("/bin/sh", "-c", "cat /proc/cpuinfo");
String result = runCommandForOutput(params);

I use this exact code and it works well for single or multiple line results. You could add an error stream handler as well.

Chicle

-1 You should use try with resources for the streams: docs.oracle.com/javase/tutorial/essential/exceptions/… – Latimer

Instead of using StringJoiner, you can use directly the stream collect(Collectors.joining(System.getProperty("line.separator"))); – Cambium

Answer 5

You might do something like this:

private static BufferedReader getOutput(Process p) {
    return new BufferedReader(new InputStreamReader(p.getInputStream()));
}

private static BufferedReader getError(Process p) {
    return new BufferedReader(new InputStreamReader(p.getErrorStream()));
}
...
Process p = Runtime.getRuntime().exec(commande);
BufferedReader output = getOutput(p);
BufferedReader error = getError(p);
String ligne = "";

while ((ligne = output.readLine()) != null) {
    System.out.println(ligne);
}

while ((ligne = error.readLine()) != null) {
 System.out.println(ligne);
}

Polyandrous

This may hang in case that process writes larger amount of data to stderr - see #16983872 – Brigandine

Answer 6

Just add .inheritIO(); to the process builder line.

IE:

ProcessBuilder pb = new ProcessBuilder(script.sh).inheritIO();

Obidiah

@MarkoBonaci see the accepted answer for that. You need to use BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream())); and then construct a StringBuilder with the output. – Electric

Note that .inheritIO() method is for java 7 and above – Electric

This redirects the process's stdOut and stdErr to Java's standard output and error; it doesn't let you capture the output as a string. – Logwood

@AndrewRueckert I am trying to figure out a way to use inheritIO and still capture the output as string. I've been researching multiple SO answers, but none of them work correctly, either buffering of the output is buildup (without inheritIO) or the InputStream is empty after the fact. Any pointers appreciated! – Evenings

@DavidFernandez I don't think you can use inheritIO in this way; I'd probably use one of the above solutions to collect the subprocess' stdout/stderr using an input stream, and, whenever I read some values from one of those streams, I'd also just use System.{out/stderr}.print to copy it to my program's output. (While being frustrated at how difficult this "simple" thing is.) – Logwood

Answer 7

For Java 7 and 8 this should work:

private String getInputAsString(InputStream is)
{
   try(java.util.Scanner s = new java.util.Scanner(is)) 
   { 
       return s.useDelimiter("\\A").hasNext() ? s.next() : ""; 
   }
}

Then in your code, do this:

String stdOut = getInputAsString(p.getInputStream());
String stdErr = getInputAsString(p.getErrorStream());

I shamelessly stole that from: How to redirect Process Builder's output to a string?

Hipolitohipp

Your link is to this post. Is that intentional? – Enscroll

it was not intentional but I wonder if someone delete the original URL and somehow my link was modified when the target was deleted. I see the community bot edited my comment so maybe that is a clue. – Hipolitohipp

Answer 8

In java 8 there is a nice lines() stream you can combine with String.join and System.lineSeparator():

    try (BufferedReader outReader = new BufferedReader(new InputStreamReader(p.getInputStream()))
    {
        return String.join(System.lineSeparator(), outReader.lines().collect(toList()));
        \\ OR using jOOλ if you like reduced verbosity
        return Seq.seq(outReader.lines()).toString(System.lineSeparator())
    }

Annunciata

Answer 9

After trying to handle different cases (handling both stderr and stdout and not blocking any of these, terminate process after timeout, properly escaping slashes, quotation marks, special characters, spaces, .... ) I gave up and found Apache Commons Exec https://commons.apache.org/proper/commons-exec/tutorial.html that seems to be doing all these things pretty well.

I do recommend everyone who needs to invoke external process in java to use Apache Commons Exec library instead of reinventing it again.

Brigandine

Thank you very much. I was trying to execute wkhtmltopdf program with arguments using ProcessBuilder. It was ok in Windows but in the Linux server it failed and I wasn't not able to get the output with BufferedReader. However everything was easy with commons-exec and it worked fine in both operating systems. – Legere

Answer 10

Another solution for Java 8:

BufferedReader stdOut = new BufferedReader(new InputStreamReader(process.getInputStream()));
String stdOutStr = stdOut.lines()
                   .collect(Collectors.joining(System.lineSeparator()));

Ribbon

Answer 11

This is for Kotlin users ending up here:

val myCommand = "java -version"
val process = ProcessBuilder()
    .command(myCommand.split(" "))
    // .directory(File("./")) // Working directory
    .redirectOutput(ProcessBuilder.Redirect.INHERIT)
    .redirectError(ProcessBuilder.Redirect.INHERIT)
    .start()
process.waitFor(60, TimeUnit.SECONDS)
val result = process.inputStream.reader().readText()
println(result)

Jackal

@Johannes_Barop Thanks for the edits. I tried to run the above but it had some problems. – Jackal

Answer 12

A complementary information to others answers, that was important in my case.
I was getting no output randomly because my process takes some time to finish.
Using process.waitFor method was no solution. What fixes my issue is using

while (process.isAlive());

before reading the inputStream.

Fayola

Answer 13

Solutions

• This code is a running example for the general solution to your question:

How to redirect Process Builder's output to a string?

• Credits go to Greg T, after trying multiple solutions to run various commands and capture their outputs Greg T's answer contained the essence of the particular solution. I hope the general example is of use for someone combining multiple requirements whilst capturing the output.
• To obtain your particular solution you can uncomment ProcessBuilder pb = new ProcessBuilder(System.getProperty("user.dir")+"/src/generate_list.sh", filename);, uncomment the line, and comment out: ProcessBuilder processBuilder = new ProcessBuilder(commands);.

Functionality

• It is a working example that executes command echo 1 and returns the output as a String.
• I also added setting a working path and an environment variable, which is not required for your particular example so you can delete it.

Usage & verification

• You can copy paste this code as a class, compile it to jar and run it.
• It is verified in WSL Ubuntu 16.04.
• Setting the workdirectory is verified by setting binaryCommand[0]="touch";and binaryCommand[1]="1";, re-compiling and running the .jar file.

Limitations

• If the pipe is full (due to a "too large" output), the code hangs.

Code

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.Map;
import java.util.StringJoiner;

public class GenerateOutput {

    /**
     * This code can execute a command and print the output accompanying that command.
     * compile this project into a .jar and run it with for example:
     * java -jar readOutputOfCommand.jar
     * 
     * @param args
     * @throws Exception 
     */
    public static void main(String[] args) throws Exception {
        boolean answerYes = false; // no yes answer to any command prompts is needed.

        // to execute a command with spaces in it in terminal, put them in an array of Strings.
        String[] binaryCommand = new String[2];

        // write a command that gives a binary output:
        binaryCommand[0] = "echo";
        binaryCommand[1] = "1";

        // pass the commands to a method that executes them
        System.out.println("The output of the echo command = "+executeCommands(binaryCommand,answerYes));
    }

    /**
     * This executes the commands in terminal. 
     * Additionally it sets an environment variable (not necessary for your particular solution)
     * Additionally it sets a working path (not necessary for your particular solution)
     * @param commandData
     * @param ansYes
     * @throws Exception 
     */
    public static String executeCommands(String[] commands,Boolean ansYes) throws Exception {
        String capturedCommandOutput = null;
        System.out.println("Incoming commandData = "+Arrays.deepToString(commands));
        File workingDirectory = new File("/mnt/c/testfolder b/");

        // create a ProcessBuilder to execute the commands in
        ProcessBuilder processBuilder = new ProcessBuilder(commands);
        //ProcessBuilder processBuilder = new ProcessBuilder(System.getProperty("user.dir")+"/src/generate_list.sh", "a");

        // this is not necessary but can be used to set an environment variable for the command
        processBuilder = setEnvironmentVariable(processBuilder); 

        // this is not necessary but can be used to set the working directory for the command
        processBuilder.directory(workingDirectory);

        // execute the actual commands
        try {

             Process process = processBuilder.start();

             // capture the output stream of the command
             BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
            StringJoiner sj = new StringJoiner(System.getProperty("line.separator"));
            reader.lines().iterator().forEachRemaining(sj::add);
            capturedCommandOutput = sj.toString();
            System.out.println("The output of this command ="+ capturedCommandOutput);

             // here you connect the output of your command to any new input, e.g. if you get prompted for `yes`
             new Thread(new SyncPipe(process.getErrorStream(), System.err)).start();
             new Thread(new SyncPipe(process.getInputStream(), System.out)).start();
            PrintWriter stdin = new PrintWriter(process.getOutputStream());

            //This is not necessary but can be used to answer yes to being prompted
            if (ansYes) {
                System.out.println("WITH YES!");
            stdin.println("yes");
            }

            // write any other commands you want here

            stdin.close();

            // this lets you know whether the command execution led to an error(!=0), or not (=0).
            int returnCode = process.waitFor();
            System.out.println("Return code = " + returnCode);
        } catch (IOException e1) {
            e1.printStackTrace();
        }
        return capturedCommandOutput;
    }


    /**
     * source: https://mcmap.net/q/242582/-using-export-in-java
     * @param processBuilder
     * @param varName
     * @param varContent
     * @return
     */
    private static ProcessBuilder setEnvironmentVariable(ProcessBuilder processBuilder){
        String varName = "variableName";
        String varContent = "/mnt/c/testfolder a/";

        Map<String, String> env = processBuilder.environment();
         System.out.println("Setting environment variable "+varName+"="+varContent);
         env.put(varName, varContent);

         processBuilder.environment().put(varName, varContent);

         return processBuilder;
    }
}


class SyncPipe implements Runnable
{   
    /**
     * This class pipes the output of your command to any new input you generated
     * with stdin. For example, suppose you run cp /mnt/c/a.txt /mnt/b/
     * but for some reason you are prompted: "do you really want to copy there yes/no?
     * then you can answer yes since your input is piped to the output of your
     * original command. (At least that is my practical interpretation might be wrong.)
     * @param istrm
     * @param ostrm
     */
    public SyncPipe(InputStream istrm, OutputStream ostrm) {
        istrm_ = istrm;
        ostrm_ = ostrm;
    }
    public void run() {

      try
      {
          final byte[] buffer = new byte[1024];
          for (int length = 0; (length = istrm_.read(buffer)) != -1; )
          {
              ostrm_.write(buffer, 0, length);                
              }
          }
          catch (Exception e)
          {
              e.printStackTrace();
          }
      }
      private final OutputStream ostrm_;
      private final InputStream istrm_;
}

Tareyn