How to mock streams in NodeJS

Question

I'm attempting to unit test one of my node-js modules which deals heavily in streams. I'm trying to mock a stream (that I will write to), as within my module I have ".on('data/end)" listeners that I would like to trigger. Essentially I want to be able to do something like this:

var mockedStream = new require('stream').readable();

mockedStream.on('data', function withData('data') {
  console.dir(data);
});

mockedStream.on('end', function() { 
  console.dir('goodbye');
});

mockedStream.push('hello world');
mockedStream.close();

This executes, but the 'on' event never gets fired after I do the push (and .close() is invalid).

All the guidance I can find on streams uses the 'fs' or 'net' library as a basis for creating a new stream (https://github.com/substack/stream-handbook), or they mock it out with sinon but the mocking gets very lengthy very quicky.

Is there a nice way to provide a dummy stream like this?

Lafollette

Answer 1

Instead of using Push, I should have been using ".emit(<event>, <data>);"

My mock code now works and looks like:

var mockedStream = new require('stream').Readable();
mockedStream._read = function(size) { /* do nothing */ };

myModule.functionIWantToTest(mockedStream); // has .on() listeners in it

mockedStream.emit('data', 'Hello data!');
mockedStream.emit('end');

Lafollette

Correction: new require('stream').Readable() (note capital R) – Embodiment

require isn't a constructor, this will work w/o the new keyword – Schelling

@Schelling require isn't the constructor here though, Readable is – Necessarily

Answer 2

There's a simpler way: stream.PassThrough

I've just found Node's very easy to miss stream.PassThrough class, which I believe is what you're looking for.

From Node docs:

The stream.PassThrough class is a trivial implementation of a Transform stream that simply passes the input bytes across to the output. Its purpose is primarily for examples and testing...

The code from the question, modified:

const { PassThrough } = require('stream');
const mockedStream = new PassThrough(); // <----

mockedStream.on('data', (d) => {
    console.dir(d);
});

mockedStream.on('end', function() {
    console.dir('goodbye');
});

mockedStream.emit('data', 'hello world');
mockedStream.end();   //   <-- end. not close.
mockedStream.destroy();

mockedStream.push() works too but as a Buffer so you'll might want to do: console.dir(d.toString());

Adelina

Answer 3

The accept answer is only partially correct. If all you need is events to fire, using .emit('data', datum) is okay, but if you need to pipe this mock stream anywhere else it won't work.

Mocking a Readable stream is surprisingly easy, requiring only the Readable lib.

  let eventCount = 0;
  const mockEventStream = new Readable({
    objectMode: true,
    read: function (size) {
      if (eventCount < 10) {
        eventCount = eventCount + 1;
        return this.push({message: `event${eventCount}`})
      } else {
        return this.push(null);
      }
    }
  });

Now you can pipe this stream wherever and 'data' and 'end' will fire.

Another example from the node docs: https://nodejs.org/api/stream.html#stream_an_example_counting_stream

Marolda

Answer 4

Building on @flacnut 's answer, I did this (in NodeJS 12+) using Readable.from() to construct a stream preloaded with data (a list of filenames):

    const mockStream = require('stream').Readable.from([
       'file1.txt',
       'file2.txt',
       'file3.txt',
    ])

In my case, I wanted to mock the stream of filenames returned by fast-glob.stream:

    const glob = require('fast-glob')
    // inject the mock stream into glob module
    glob.stream = jest.fn().mockReturnValue(mockStream)

In the function being tested:

  const stream = glob.stream(globFilespec)
  for await (const filename of stream) {
    // filename = file1.txt, then file2.txt, then file3.txt
  }

Works like a charm!

Trimurti

Answer 5

Here's a simple implementation which uses jest.fn() where the goal is to validate what has been written to the stream created by fs.createWriteStream(). The nice thing about jest.fn() is that although the calls to fs.createWriteStream() and stream.write() are inline in this test function, these functions don't need to be called directly by the test.

    const fs = require('fs');
    const mockStream = {}
    
    test('mock fs.createWriteStream with mock implementation', async () => {
      const createMockWriteStream = (filename, args) => {
        return mockStream;
      }

      mockStream3.write = jest.fn();
      fs.createWriteStream = jest.fn(createMockWriteStream);
  
      const stream = fs.createWriteStream('foo.csv', {'flags': 'a'});
      await stream.write('foobar');
      expect(fs.createWriteStream).toHaveBeenCalledWith('foo.csv', {'flags': 'a'});
      expect(mockStream.write).toHaveBeenCalledWith('foobar');
  })

Mollusc

Answer 6

I had to set result as Buffer:

const str = '{ "fileName": "file.pdf" }';
const buf = new Buffer(str.length);

for (let i = 0; i < str.length ; i++) {
  buf[i] = str.charCodeAt(i);
}


let eventCount = 0;
const mockedStream = new Readable({
  objectMode: true,
  read: function () {
    if (eventCount < 1) {
      eventCount = eventCount + 1;
      return this.push(buf);
    } else {
      return this.push(null);
    }
  }
});

Orianna