Node.js child processes and pipes - OSX vs Ubuntu

Question

I am trying to get two long running node.js processes to communicate - a parent and a child - using pipes and Node's child-process module. I want the child to be able to send data back to the parent asynchronously, and I was hoping to use a pipe to do so.

Here's a simplified version of my code:

Parent:

cp = require('child_process')
es = require('event-stream')

child = cp.spawn('coffee', ['child.coffee'], {stdio: [null, null, null, 'pipe']})

so = child.stdout.pipe(es.split())
p3 = child.stdio[3].pipe(es.split())

so.on 'data', (data) ->
  console.log('stdout: ' + data)

child.stderr.on 'data', (data) ->
  console.log('stderr: ' + data);

p3.on 'data', (data) ->
  console.log('stdio3: ' + data);

child.on 'close', (code) ->
  console.log('child process exited with code ' + code)

child.stdin.write "a message from your parent", "utf8"

Child:

fs = require('fs')

p3 = fs.createWriteStream('/dev/fd/3', {encoding: 'utf8'})

process.stdin.on 'data', (data) ->
    p3.write "hello #{process.pid} - #{data}\n", 'utf8'
    process.stdout.write "world #{process.pid} - #{data}\n", 'utf8'
    p3.end()
    process.exit(0)

process.stdin.on 'end', (data) ->
    console.log "end of stdin"
    p3.end()
    process.exit(0)

process.stdin.setEncoding('utf8')
process.stdin.resume()

The code works on OSX 10.9, but fails to run on a Ubuntu box. I have tried running it under both Ubuntu 12.04 and 14.04. I am running Node 10.2x.

/dev/fd/ under Ubuntu is symbolically linked to /proc/self/fd/ so I believe my child process is opening the right file.

The output from running the parent on Ubuntu is as follows:

$ coffee parent.coffee 
stderr: 

stderr: events.js:72

stderr:         throw er; // Unhandled 'error' event

stderr:  
stderr:  
stderr:  
stderr:  
stderr:           ^

stderr: Error: UNKNOWN, open '/dev/fd/3'




events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: read ECONNRESET
  at errnoException (net.js:901:11)
  at Pipe.onread (net.js:556:19)

I would expect to see (and do on a OSX box):

$ coffee parent.coffee 
stdio3: hello 21101 - a message from your parent
stdout: world 21101 - a message from your parent
stdio3: 
stdout: 
child process exited with code 0

It is possible to communicate with the child using the command line also on Ubuntu, so the problem is likely in the parent when spawning the child process:

$ echo foo | coffee child.coffee 3>&1
hello 3077 - foo

world 3077 - foo

I have tried to investigate the kernel calls that node makes using strace, but couldn't make much sense of the output.

Wadai

What is the output of ps aux | grep node when both processes are running? – Bristle

@ChrisLacko The command coffee parent.coffee crashes immediately. On OSX it prints what it is supposed to and then exits. I can add the errors that I see when I run it to the post. – Wadai

Answer 1

I figured it out myself. The error was in the child. Ubuntu linux is more strict when it comes to opening files that are already open, the line:

p3 = fs.createWriteStream('/dev/fd/3', {encoding: 'utf8'})

was throwing an error. The file descriptor 3 is already open when the child runs, so the code should look as follows:

Child:

fs = require('fs')

# parent opens the file descriptor 3 when spawning the child (and closes it when the child returns)
fd3write = (s) ->
    b = new Buffer(s)
    fs.writeSync(3,b,0,b.length)

process.stdin.on 'data', (data) ->
    fd3write "p3 #{process.pid} - #{data}\n"
    process.stdout.write "so #{process.pid} - #{data}\n", 'utf8'
    process.exit(0)

process.stdin.on 'end', (data) ->
    console.log "end of stdin"
    process.exit(0)

process.stdin.setEncoding('utf8')
process.stdin.resume()

I hope this will be of help to someone else.

To use a pipe instead of stdin to send messages from the parent to the child this link might be of use: child-process-multiple-file-descriptors.

Wadai