node.js: readSync from stdin?

Question

Is it possible to synchronously read from stdin in node.js? Because I'm writing a brainfuck to JavaScript compiler in JavaScript (just for fun). Brainfuck supports a read operation which needs to be implemented synchronously.

I tried this:

const fs = require('fs');
var c = fs.readSync(0,1,null,'utf-8');
console.log('character: '+c+' ('+c.charCodeAt(0)+')');

But this only produces this output:

fs:189
  var r = binding.read(fd, buffer, offset, length, position);
              ^
Error: EAGAIN, Resource temporarily unavailable
    at Object.readSync (fs:189:19)
    at Object.<anonymous> (/home/.../stdin.js:3:12)
    at Module._compile (module:426:23)
    at Module._loadScriptSync (module:436:8)
    at Module.loadSync (module:306:10)
    at Object.runMain (module:490:22)
    at node.js:254:10

Piliform

Save yourself time and use a well maintained npm library that abstracts reading from stdin, npmjs.com/package/get-stdin. – Stephenson

do you mean "synchronous" or "blocking read"? a blocking read of one byte would wait forever when there is no input data, but this is not possible in javascript. as a workaround, you can use child_process.spawnSync with an external command like dd ibs=1 count=1 status=none or head -c1, see also How to timeout an fs.read in node.js? or Can I read a single character from stdin in POSIX shell? – Defensible

Answer 1

I've no idea when this showed up but this is a helpful step forward: http://nodejs.org/api/readline.html

var readline = require('readline');

var rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
  terminal: false
});

rl.on('line', function (cmd) {
  console.log('You just typed: '+cmd);
});

Now I can read line-at-a-time from stdin. Happy days.

Degroot

Nice; just a heads-up: the readline module is still classified as 2 - Unstable as of Node.js v0.10.4. – Willumsen

@Willumsen I believe the 2 - Unstable means the API is not firm and is subject to change. nodejs.org/api/documentation.html#documentation_stability_index. I'm not sure what it means with respect to stability for use. – Cola

@AlainO'Dea: Thanks; to quote the page you linked to: 'Backwards-compatibility will be maintained if reasonable.' I read this as: 'probably won't change, but we reserve the right to do so', which from a user's perspective translates to: 'feature is here to stay, likely with its present API, but there's a slight chance of a future breaking API change'. – Willumsen

This technique isn't synchronous. – Sunbathe

@BarryKelly rl.prompt can awaited; or it can actually be synchronized. – Upanchor

Not sure why this became the accepted answer as it doesn't answer the question, which is to synchronously read from stdin. This is not synchronous. @Upanchor Awaiting something still doesn't make it synchronous, it just makes the code look synchronous-like which is something very different. It still lets other things happen in the background. Which in many cases is what you want but in some cases is something that you might really not want. – Alimony

@Alimony You're right, it isn't synchronous. It turns out, the OP didn't need synchronous; they just needed to get past an error message. BTW, synchronous code doesn't require other threads not to run; it just means that the thread in question waits (blocks) for input (ie. synchronizes with the input) before continuing its processing. Since JS only allows one thread to run at a time, this is as close as you can get to synchronous without blocking other threads, and it's probably better for the OP's use case, anyhow. – Upanchor

@Alimony ...and my first comment has an or between the words await and synchronized. Maybe that link I provided will help you achieve your task. – Upanchor

Answer 2

Have you tried:

fs=require('fs');
console.log(fs.readFileSync('/dev/stdin').toString());

However, it will wait for the ENTIRE file to be read in, and won't return on \n like scanf or cin.

Nonessential

That's not good enough because it needs to be an interactive prompt. – Piliform

This answer saved me a bunch of refactoring time - thanks! It looks like it won't work on Windows. But I'm not too concerned about that. – Airs

@Piliform If you want it to block on every line, you will need to implement your own C+ wrapper around getline() or some such function – Nonessential

Very convenient, but there are 2 caveats: this solution (a) doesn't work on Windows (as @JesseHallett stated), and (b) exhibits non-standard behavior with interactive stdin input: instead of processing the interactive input line by line, the readFileSync() call blocks until all lines has been received (implied by @dhruvbird's disclaimer, but it's worth stating explicitly). – Willumsen

caveat 3 this seems to only read the first 65536 characters of available input – Procuration

FYI about @Armand's third caveat: appears to have been fixed circa io.js v1.6-ish: github.com/nodejs/node/pull/1074 – Branks

Re Windows support: in v5+, using 0 instead of /dev/stdin makes the approach work on Windows too, but as of v9.11.1 there is a bug when there is no input or stdin is closed. – Willumsen

Works for me with >1MiB of input. Windows apparently solved by using 0 as file descriptor ID. The "non-standard behavior" is perfectly standard behavior for a program that, as the answer says, reads data from stdin rather than trying to be an interactive query-response program. I see no caveats here, using Linux with apt install nodejs -> version 12.22.12 – Rettke

Answer 3

After fiddling with this for a bit, I found the answer:

process.stdin.resume();
var fs = require('fs');
var response = fs.readSync(process.stdin.fd, 100, 0, "utf8");
process.stdin.pause();

response will be an array with two indexes, the first being the data typed into the console and the second will be the length of the data including the newline character.

It was pretty easy to determine when you console.log(process.stdin) which enumerates all of the properties including one labeled fd which is of course the name of the first parameter for fs.readSync()

Enjoy! :D

Emulate

Even on v0.7.5-pre that gives the same "Error: UNKNOWN, unknown error" as a plain fs.readSync from STDIN. Which version of node and OS did that work on? – Degroot

@Degroot I just double checked the code and it worked for me on Windows7 and v0.6.7. I'm setting up 0.6.12 on my linux box right now so I'll let you know what I get there when it's done – Emulate

@Degroot - yeah looks like there is a bug in the underlying dependency libs for file reading... well not a bug, just a caveat not accounted for. I'm really not a strong c developer but it looks like the open() call on stdin will fail if it's already opened. To work around this I believe they need to dup() the handle if the handle is a 0 or 1 and dup2() the handle back after completion. But then again I could be woefully incorrect :D. I'd open a ticket on github and let some real c devs give you the right answer. – Emulate

This approach still works in principle (with limitations), but the code in this answer no longer works as of node.js v0.10.4, because the interfaces have changed; see my answer. – Willumsen

Answer 4

An updated version of Marcus Pope's answer that works as of node.js v0.10.4:

Please note:

• In general, node's stream interfaces are still in flux (pun half-intended) and are still classified as 2 - Unstable as of node.js v0.10.4.
• Different platforms behave slightly differently; I've looked at OS X 10.8.3 and Windows 7: the major difference is: synchronously reading interactive stdin input (by typing into the terminal line by line) only works on Windows 7.

Here's the updated code, reading synchronously from stdin in 256-byte chunks until no more input is available:

var fs = require('fs');
var BUFSIZE=256;
var buf = new Buffer(BUFSIZE);
var bytesRead;

while (true) { // Loop as long as stdin input is available.
    bytesRead = 0;
    try {
        bytesRead = fs.readSync(process.stdin.fd, buf, 0, BUFSIZE);
    } catch (e) {
        if (e.code === 'EAGAIN') { // 'resource temporarily unavailable'
            // Happens on OS X 10.8.3 (not Windows 7!), if there's no
            // stdin input - typically when invoking a script without any
            // input (for interactive stdin input).
            // If you were to just continue, you'd create a tight loop.
            throw 'ERROR: interactive stdin input not supported.';
        } else if (e.code === 'EOF') {
            // Happens on Windows 7, but not OS X 10.8.3:
            // simply signals the end of *piped* stdin input.
            break;          
        }
        throw e; // unexpected exception
    }
    if (bytesRead === 0) {
        // No more stdin input available.
        // OS X 10.8.3: regardless of input method, this is how the end 
        //   of input is signaled.
        // Windows 7: this is how the end of input is signaled for
        //   *interactive* stdin input.
        break;
    }
    // Process the chunk read.
    console.log('Bytes read: %s; content:\n%s', bytesRead, buf.toString(null, 0, bytesRead));
}

Willumsen

This is the only way I've been able to capture STDIN in its entirety when input is lengthy. – Whiffen

while(true)? break? If bytesRead === 0 is your condition, why are you using break statements? – Rascal

your condition is not true. Your condition is while there are bytes in stdin and if there's no error while processing. That's what I'm saying. It's spaghetti code. – Rascal

you don't need to introduce a variable for that, just negate the if that holds the break statement. You may introduce an error variable that is TRUE when any error is found while reading. Yes, it's worth, your code is your docs. while(true) doesn't say anything to me. while(bytesRead != 0 && !error) it does. – Rascal

@Sebastian: Taking a step back: The code in the answer is well-commented and discusses the issues that matter for the problem at hand. Whether your concerns regarding spaghetti code have merit or not is incidental to the problem, and this is not the place to discuss them: Your comments are creating a distraction for future readers. I've deleted my previous comments to minimize the distraction. – Willumsen

@Rascal maybe such rough edges will dissuade people from just copy/pasting the answer verbatim.. which legal depts hate at software companies! :) The answer is provided to satisfy the ask, not a code review! – Nordgren

Answer 5

I found a library that should be able to accomplish what you need: https://github.com/anseki/readline-sync

Huddle

Why is this so far down? This is the best answer! It accomplishes what OP wants in 1 line of code! – Coliseum

I found it useful! I used it in a few projects and I can confirm it does what it says it does :) – Huddle

This doesn't work with piped input. foo | bar where bar uses readline-sync will try to read from the terminal, not from stdin. – Sunbathe

Answer 6

Here is the implementation with `async await`. In the below code, the input is taken from standard input and after receiving data the standard input is stopped waiting for data by using `process.stdin.pause();`.

process.stdin.setEncoding('utf8');

// This function reads only one line on console synchronously. After pressing `enter` key the console will stop listening for data.
function readlineSync() {
    return new Promise((resolve, reject) => {
        process.stdin.resume();
        process.stdin.on('data', function (data) {
            process.stdin.pause(); // stops after one line reads
            resolve(data);
        });
    });
}

// entry point
async function main() {
    let inputLine1 = await readlineSync();
    console.log('inputLine1 = ', inputLine1);
    let inputLine2 = await readlineSync();
    console.log('inputLine2 = ', inputLine2);
    console.log('bye');
}

main();

Derbent

Please explain how and why your code works to help people who face similar problems in the future – Brasserie

Thank you so much, that is exactly what I needed. Confirmed to work in Node 10 – Cathrinecathryn

This also helped me. It is a good answer but you should explain what and why are you doing so that anyone can understand. – Exhaustless

This solution is not synchronous. – Radiolarian

Answer 7

Important: I've just been informed by a Node.js contributor that .fd is undocumented and serves as a means for internal debugging purposes. Therefore, one's code should not reference this, and should manually open the file descriptor with fs.open/openSync.

In Node.js 6, it's also worth noting that creating an instance of Buffer via its constructor with new is deprecated, due to its unsafe nature. One should use Buffer.alloc instead:

'use strict';

const fs = require('fs');

// small because I'm only reading a few bytes
const BUFFER_LENGTH = 8;

const stdin = fs.openSync('/dev/stdin', 'rs');
const buffer = Buffer.alloc(BUFFER_LENGTH);

fs.readSync(stdin, buffer, 0, BUFFER_LENGTH);
console.log(buffer.toString());
fs.closeSync(stdin);

Also, one should only open and close the file descriptor when necessary; doing this every time one wishes to read from stdin results in unnecessary overhead.

Circus

Important: .fd is currently documented. – Bickel

Answer 8

The following code reads sync from stdin. Input is read up until a newline / enter key. The function returns a string of the input with line feeds (\n) and carriage returns (\r) discarded. This should be compatible with Windows, Linux, and Mac OSX. Added conditional call to Buffer.alloc (new Buffer(size) is currently deprecated, but some older versions lack Buffer.alloc.

function prompt(){
    var fs = require("fs");

    var rtnval = "";

    var buffer = Buffer.alloc ? Buffer.alloc(1) : new Buffer(1);

    for(;;){
        fs.readSync(0, buffer, 0, 1);   //0 is fd for stdin
        if(buffer[0] === 10){   //LF \n   return on line feed
            break;
        }else if(buffer[0] !== 13){     //CR \r   skip carriage return
            rtnval += new String(buffer);
        }
    }

    return rtnval;
}

Livvi

The fs.readSync call will throw after EOF, so be sure to wrap it in a try/catch. – Lewert

also note that this is not a "blocking read". add process.stdout.write("."); in the for(;;) loop and run printf "" | ./read.js - this will flood your terminal with dots, and the loop will consume 100% cpu time – Defensible

Answer 9

function read_stdinSync() {
    var b = new Buffer(1024)
    var data = ''

    while (true) {
        var n = fs.readSync(process.stdin.fd, b, 0, b.length)
        if (!n) break
        data += b.toString(null, 0, n)
    }
    return data
}

Kriss

This was a treasure. Thanks for sharing. I am gonna implement in my open source node module. Will come back to share the link to the node module after some testing. Thanks! – Volcanic

Here is the new module based on your idea:- line-reader – Volcanic

@VikasGautam thank you, well done. Do you read entire stdin at once or yield lines as they come? – Kriss

Iine-reader is a generator function that reads 64 * 1024 bytes once until everything is read from a file or stdin and spits single line with each .next() call or iteration. Check the index.ts. I think, I should also make defaultChunkSize as a param from user. I am gonna push an update. – Volcanic

Answer 10

I used this workaround on node 0.10.24/linux:

var fs = require("fs")
var fd = fs.openSync("/dev/stdin", "rs")
fs.readSync(fd, new Buffer(1), 0, 1)
fs.closeSync(fd)

This code waits for pressing ENTER. It reads one character from line, if user enters it before pressing ENTER. Other characters will be remained in the console buffer and will be read on subsequent calls to readSync.

Preussen

this is not a "blocking read" see printf "" | ./read.js – Defensible

Answer 11

I wrote this module to read one line at a time from file or stdin. The module is named as line-reader which exposes an ES6 *Generator function to iterate over one line at a time. here is a code sample(in TypeScript) from readme.md.

import { LineReader } from "line-reader"

// FromLine and ToLine are optional arguments
const filePathOrStdin = "path-to-file.txt" || process.stdin
const FromLine: number = 1 // default is 0
const ToLine: number = 5 // default is Infinity
const chunkSizeInBytes = 8 * 1024 // default is 64 * 1024

const list: IterableIterator<string> = LineReader(filePathOrStdin, FromLine, ToLine, chunkSizeInBytes)

// Call list.next to iterate over lines in a file
list.next()

// Iterating using a for..of loop
for (const item of list) {
   console.log(item)
}

Apart from above code, you can also take a look at src > tests folder in the repo.

Note:-
line-reader module doesn't read all stuff into memory instead it uses generator function to generate lines async or sync.

Volcanic