nodejs child_process.spawnSync or child_process.spawn wrapped in yieldable generator which returns output

Question

since a while i am trying to reach something that doesn't work out for me so far.

With nodejs, i like to run a interactive sh-command and work with the sh-command output after the command has exited. i like to write a yieldable generator-function that wraps the running of the interactive shell command and returns the output of the shell command.

Approach 1: shelljs

• shelljs
• I had some success with shelljs, but at some point it wont run further.
• Question 1: is it possible to get shelljs to the point where i can inherit stdio and make the shelljs function yieldable?

Approach 2: child_process.spawnSync

• child_process.spawnSync
• at last i discovered child_process.spawnSync and was happy that, at least i can run interactive sh-commands without problems with options: { stdio: 'inherit' }
• but i haven't found out how to get back the output of child_process.spawnSync.
• Question 2: How to wrap spawnSync into a generator function that returns the child_process's output?

Approach 3: co-child-process

• i also tried co-child-process.
• it seems to run, but not interactive with stdio. there is a issue regarding this, i dont really understand.
• Question 3: could someone explain me/ post a example how co-child-process will work with stdio inherit.

Approach 4: promisify child_process.spawn() with bluebird

• i opened an issue on bluebird if child_process.spawn() is promisifiable

So my question at all. Can someone post me an example of how to run a interactive shell command that can be wrapped in a yieldable generator function that returns the output of the shell command? i am open for new approaches.

I created a npm module which is available on github where you can fork it and contribute.

thx in advance.

Betoken

Its not clear from the question what exactly you want to yield. Just the spawn command? Or would you like to yield read/write commands for the process stdin/stdout? Can you describe the original problem you're trying to solve? – Galbreath

i like to yield the spawn command and get all the output that possibly occured. the original problem i had is the following. i try to run different interactive shell commands serially. later commands could be dependent on the result (output) of a predecessor command, so i need the predecessors output. – Betoken

You cannot get all the output if you want interactivity. You will have to send separate read/write commands. Furthermore, the process may be stalled while working on giving you the next chunk of output instead of waiting for input, and there is no sure way to differentiate between those two states. In short, I don't think you can solve your problem with yield in the general case (your specific case may be solvable if you share more info about the concrete process you're trying to control and its output) – Galbreath

See also: github.com/chjj/pty.js which is a better starting point than child_process as it can emulate the behaviour of an interactive terminal. Edit: I think I understand what you want to do - you want to include another program's interactive session within your own program? – Galbreath

Answer 1

I found the following which works on v5.4.1 . In the docs NodeJS Child Process it mentions the option encoding which has a default of 'buffer'. If you set this option to 'utf8', then instead of a buffer you get a string back with the results. You can get a string back from spawnSync because it is synchronous and blocks execution until the command completes. Here's a working example of a script which does an 'ls -l /usr' command and gets the output as a string object:

#!/usr/bin/env node

var cp = require('child_process');

var ls = cp.spawnSync('ls', ['-l', '/usr'], { encoding : 'utf8' });
// uncomment the following if you want to see everything returned by the spawnSync command
// console.log('ls: ' , ls);
console.log('stdout here: \n' + ls.stdout);

When you run it you get the following:

stdout here:
total 68
drwxr-xr-x   2 root root 36864 Jan 20 11:47 bin
drwxr-xr-x   2 root root  4096 Apr 10  2014 games
drwxr-xr-x  34 root root  4096 Jan 20 11:47 include
drwxr-xr-x  60 root root  4096 Jan 20 11:47 lib
drwxr-xr-x  10 root root  4096 Jan  4 20:54 local
drwxr-xr-x   2 root root  4096 Jan  6 01:30 sbin
drwxr-xr-x 110 root root  4096 Jan 20 11:47 share
drwxr-xr-x   6 root root  4096 Jan  6 00:34 src

The docs tell you what you can get back on the object in addition to stdout . If you want to see all the properties on the return object, uncomment the console.log (Warning: there's a LOT of stuff :) ).

Salvidor

The spawnSync() seems to be output everything at once after finish process, unlike spawn(). Is this correct? – Satisfaction

@Satisfaction yes. And it has a limited buffer 1MB buffer: https://mcmap.net/q/559741/-spawnsync-bin-sh-enobufs/… We can read line by line instead with spawn + readline. – Wearproof

Answer 2

To iterate line by line as soon as lines become available you can:

#!/usr/bin/env node

const childProcess = require('child_process')
const readline = require('readline')

const p = childProcess.spawn('ls', ['somedir'])
;(async function() {
const rl = readline.createInterface({ input: p.stdout })
for await (const line of rl) {
  console.log('read: ' + line)
}
})()

This is a good idea when your output is line oriented, as it allows you to process data that is larger than memory, and it allows you to start processing early stdout lines immediately without waiting for the last ones. It also overcomes the default spawnSync 1MB buffer limit.

I've provided more a more detailed example of this at: spawnSync /bin/sh ENOBUFS

Wearproof