To reliably start a sub-process you need to handle the output streams at same time or the process will block when either STDOUT or STDERR is not consumed when they fill to the default buffer limit.
You can demonstrate this issue by these test commands which write large amount of data to STDOUT and STDERR at same pace. If your app does not keep up with reading from both of these streams then the sub-process will freeze / deadlock:
// WINDOWS:
String[] commands = {"cmd.exe", "/c", "FOR /L %X IN (1, 1, 10000) DO echo Hello STDOUT %X && echo Hello STDERR %X 1>&2"};
// Linux / Unix style OS
String[] commands = {"/bin/bash", "-c", "for i in {1..10000} ; do echo Hello STDERR $i 1>&2 ; echo Hello STDOUT $i; done"};
You can can avoid the problem by using ProcessBuilder
which gives better control of where output streams go, and prevent deadlock situation by calling pb.redirectErrorStream(true)
or pb.inheritIO()
or redirect either of STDOUT / STDERR to File
using pb.redirectOutput/Error(file)
/ or use different threads for reading from STDOUT and STDERR.
Here is a simple example of how to handle launch which could be used in place of Runtime.exec()
and sends STDOUT(/STDERR) to any stream you pass in, and which avoids the deadlock situation:
// Example:
start(command, null, System.out, null);
// or
start(command, null, System.out, System.err);
// Don't forget to close streams you pass in - if appropriate
public static int start(String[] cmd, byte[] stdin, OutputStream stdout, OutputStream stderr)
throws IOException, InterruptedException
{
Objects.requireNonNull(cmd);
Objects.requireNonNull(stdout);
System.out.println("start "+Arrays.toString(cmd));
// Launch and wait:
ProcessBuilder pb = new ProcessBuilder(cmd);
if (stderr == null) {
pb.redirectErrorStream(true); // No STDERR => merge to STDOUT
}
Process p = pb.start();
// Consumes STDERR at same time as STDOUT, not doing this large streams can block I/O in the sub-process
Thread bg = null;
if (stderr != null) {
Runnable task = () -> {
try(var from = p.getErrorStream()) {
from.transferTo(stderr);
} catch(IOException io) {
throw new UncheckedIOException(io);
}
};
bg = new Thread(task, "STDERR");
bg.start();
}
// Send STDIN if required, and close STDIN stream
// NOTE!!! a huge input stream can lock up STDOUT/STDERR readers, you may need a background thread here too
try(OutputStream os = p.getOutputStream()) {
if (stdin != null) os.write(stdin);
}
// Move STDOUT to the output stream
try(var stdo = p.getInputStream()) {
stdo.transferTo(stdout);
}
int rc = p.waitFor();
if (bg != null) {
bg.join();
}
System.out.println("start "+Arrays.toString(cmd));
System.out.println("Exit "+p.pid()+" CODE "+rc +' '+(rc == 0 ? "OK":"**** ERROR ****")+" "+(stderr == null ? "STDERR>OUT":""));
return rc;
}