How to write a string to Scala Process?

Question

I start and have running a Scala process.

    val dir = "/path/to/working/dir/"
    val stockfish = Process(Seq("wine", dir + "stockfish_8_x32.exe"))
    val logger = ProcessLogger(printf("Stdout: %s%n",  _))
    val stockfishProcess = stockfish.run(logger, connectInput = true)

The process reads from and writes to standard IO (console). How can I send a string command to the process if it's been already started?

Scala process API has ProcessBuilder which has in turn bunch of useful methods. But ProcessBuilder is used before a process starts to compose complex shell commands. Also Scala has ProcessIO to handle input or output. I don't need it too. I just need to send message to my process.

In Java I would do something like this.

        String dir = "/path/to/working/dir/";
        ProcessBuilder builder = new ProcessBuilder("wine", dir + "stockfish_8_x32.exe");
        Process process = builder.start();

        OutputStream stdin = process.getOutputStream();
        InputStream stdout = process.getInputStream();

        BufferedReader reader = new BufferedReader(new InputStreamReader(stdout));
        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));

        new Thread(() -> {
            try {
                String line;
                while ((line = reader.readLine()) != null) {
                    System.out.println("Stdout: " + line);
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }).start();

        Thread.sleep(5000); // it's just for example
        writer.write("quit");  // send to the process command to stop working
        writer.newLine();
        writer.flush();

It works quite well. I start my process, get InputStream and OutputStream from it, and use the streams to interact with the process.

It appears Scala Process trait provides no ways to write to it. ProcessBuilder is useless after process run. And ProcessIO is just for IO catching and handling.

Are there any ways to write to Scala running process?

UPDATE:

I don't see how I may use ProcessIO to pass a string to running process. I did the following.

import scala.io.Source
import scala.sys.process._

object Sample extends App {

   def out = (output: java.io.OutputStream) => {
      output.flush()
      output.close()
   }

   def in = (input: java.io.InputStream) => {
      println("Stdout: " + Source.fromInputStream(input).mkString)
      input.close()
   }

   def go = {
      val dir = "/path/to/working/dir/"
      val stockfishSeq = Seq("wine", dir + "/stockfish_8_x32.exe")
      val pio = new ProcessIO(out, in, err => {})
      val stockfish = Process(stockfishSeq)
      stockfish.run(pio)

      Thread.sleep(5000)
      System.out.write("quit\n".getBytes)
      pio.writeInput(System.out) // "writeInput" is function "out" which I have passed to conforming ProcessIO instance. I can invoke it from here. It takes OutputStream but where can I obtain it? Here I just pass System.out for example.
   }
   go
 }

Of course it does not work and I failed to understand how to implement functionality as in my Java snippet above. It would be great to have advice or snippet of Scala code clearing my issue.

Sohn

I think I misunderstood your question, so I deleted my answer. You should probably specify which version of Scala you're using (I think subprocess-related APIs have changed recently). I'm confused why ProcessIO doesn't work for you. It looks like your Java code is just using stdin/stdout, which is also possible with ProcessIO. Could you clarify? – Streamlined

@Streamlined Yes, it is quite simple in Java. Java "Process" class provides methods returning IO streams that I can use. Scala "Process" trait has only three methods: destroy, exitValue, and isAlive. ProcessIO class has function-parameter "writeInput" which may be passed and called afterwards directly. As I guess it is just kind of listener for input events, but how to originate this "event"?. I use the last Scala version - 2.12.1. – Sohn

Answer 1

I think the documentation around Scala processes (specifically the usage and semantics of ProcessIO) could use some improvement. The first time I tried using this API, I also found it very confusing, and it took some trial and error to get my subprocess i/o working correctly.

I think seeing a simple example is probably all you really need. I'll do something really simple: invoking bc as a subprocess to do some trivial computations, and then printing the answers to my stdout. My goal is to do something like this (but from Scala rather than from my shell):

$ printf "1+2\n3+4\n" | bc
3
7

Here's how I'd do it in Scala:

import scala.io.Source
import scala.sys.process._

object SimpleProcessExample extends App {

  def out = (output: java.io.OutputStream) => {
    output.flush()
    output.close()
  }

  def in = (input: java.io.InputStream) => {
    println("Stdout: " + Source.fromInputStream(input).mkString)
    input.close()
  }

  // limit scope of any temporary variables
  locally {
    val calcCommand = "bc"
    // strings are implicitly converted to ProcessBuilder
    // via scala.sys.process.ProcessImplicits.stringToProcess(_)
    val calcProc = calcCommand.run(new ProcessIO(
      // Handle subprocess's stdin
      // (which we write via an OutputStream)
      in => {
        val writer = new java.io.PrintWriter(in)
        writer.println("1 + 2")
        writer.println("3 + 4")
        writer.close()
      },
      // Handle subprocess's stdout
      // (which we read via an InputStream)
      out => {
        val src = scala.io.Source.fromInputStream(out)
        for (line <- src.getLines()) {
          println("Answer: " + line)
        }
        src.close()
      },
      // We don't want to use stderr, so just close it.
      _.close()
    ))

    // Using ProcessBuilder.run() will automatically launch
    // a new thread for the input/output routines passed to ProcessIO.
    // We just need to wait for it to finish.

    val code = calcProc.exitValue()

    println(s"Subprocess exited with code $code.")

  }
}

Notice that you don't actually call any of the methods of the ProcessIO object directly because they're automatically called by the ProcessBuilder.

Here's the result:

$ scala SimpleProcessExample
Answer: 3
Answer: 7
Subprocess exited with code 0.

If you wanted interaction between the input and output handlers to the subprocess, you can use standard thread communication tools (e.g., have both close over an instance of BlockingQueue).

Streamlined

Thank you for your example. Very illustrative. However in your snippet you know what exactly you want to calculate at the moment of creation of process, and there is no problem to move all logic to method "in". – Sohn

Just imagine your calculator is waiting for an user enters some expression which the calculator should processes then. There will be nothing to write in "in" function. You don't know what exactly user types. If you had OutputStream instance, like in Java, you could write to it any time whenever an user types something. But you don't have. And this is my problem. I can start process but fail to write to it during its work. – Sohn

Answer 2

Here is an example of obtaining input and output streams from a process, which you can write to and read from after the process starts:

object demo {
  import scala.sys.process._

  def getIO = {
    // create piped streams that can attach to process streams:
    val procInput = new java.io.PipedOutputStream()
    val procOutput = new java.io.PipedInputStream()
    val io = new ProcessIO(
      // attach to the process's internal input stream
      { in =>
        val istream = new java.io.PipedInputStream(procInput)
        val buf = Array.fill(100)(0.toByte)
        var br = 0
        while (br >= 0) {
          br = istream.read(buf)
          if (br > 0) { in.write(buf, 0, br) }
        }
        in.close()
      },
      // attach to the process's internal output stream
      { out =>
        val ostream = new java.io.PipedOutputStream(procOutput)
        val buf = Array.fill(100)(0.toByte)
        var br = 0
        while (br >= 0) {
          br = out.read(buf)
          if (br > 0) { ostream.write(buf, 0, br) }
        }
        out.close()
      },
      // ignore stderr
      { err => () }
    )
    // run the command with the IO object:
    val cmd = List("awk", "{ print $1 + $2 }")
    val proc = cmd.run(io)

    // wrap the raw streams in formatted IO objects:
    val procO = new java.io.BufferedReader(new java.io.InputStreamReader(procOutput))
    val procI = new java.io.PrintWriter(procInput, true)
    (procI, procO)
  }
}

Here's a short example of using the input and output objects. Note that it's hard to guarantee that the process will receive it's input until you close the input streams/objects, since everything is piped, buffered, etc.

scala> :load /home/eje/scala/input2proc.scala
Loading /home/eje/scala/input2proc.scala...
defined module demo

scala> val (procI, procO) = demo.getIO
procI: java.io.PrintWriter = java.io.PrintWriter@7e809b79
procO: java.io.BufferedReader = java.io.BufferedReader@5cc126dc

scala> procI.println("1 2")

scala> procI.println("3 4")

scala> procI.println("5 6")

scala> procI.close()

scala> procO.readLine
res4: String = 3

scala> procO.readLine
res5: String = 7

scala> procO.readLine
res6: String = 11

scala> 

In general, if you are managing both input and output simultaneously in the same thread, there is the potential for deadlock, since either read or write can block waiting for the other. It is safest to run input logic and output logic in their own threads. With these threading concerns in mind, it is also possible to just put the input and output logic directly into the definitions { in => ... } and { out => ... }, as these are both run in separate threads automatically

Amur

Thank you for this. I needed a piped streams example. I had to add ostream.close to the ProcessIO out function, otherwise I received IOException: Write end dead. – Clevis

In further testing, sometimes I would get a Pipe not connected error. I'm pretty sure this can happen if I try to read procO before the out function has a chance to run. I solved that by creating the PipedOutputStream and connecting it earlier: val procOutput = new java.io.PipedInputStream(); val pop = new java.io.PipedOutputStream(procOutput) then using pop in the out function rather than ostream. – Clevis

Answer 3

var outPutStream: Option[OutputStream] = None
val io = new ProcessIO(
  { outputStream =>
    outPutStream = Some(outputStream)
  },
  Source.fromInputStream(_).getLines().foreach(println),
  Source.fromInputStream(_).getLines().foreach(println)
)
command run io

val out = outPutStream.get
out.write("test" getBytes())

You can get an InputStream in the same way.

Putupon

Answer 4

I haven't actually tried this, but the documentation says that you can use a instance of ProcessIO to handle the Process's input and output in a manner similar to what you would do in Java.

Kootenay