What's the difference between .pipe and pipeline on streams

Question

I found two different ways to pipe streams in node.js

Well known .pipe() method of a stream

https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options

and standalone function for streams

https://nodejs.org/api/stream.html#stream_stream_pipeline_streams_callback

Which one should I use and what are the benefits between those two?

Schizopod

Answer 1

TL;DR - You better want to use pipeline

What's pipeline?

From the docs: A module method to pipe between streams forwarding errors and properly cleaning up and provide a callback when the pipeline is complete.

What's the motivation for using pipeline?

❌ Let's take a look at the following code:

const { createReadStream } = require('fs');
const { createServer } = require('http');
const server = createServer(
  (req, res) => {
    createReadStream(__filename).pipe(res);
  }
);

server.listen(3000);

What's wrong here? If the response will quit or the client closes the connection - then the read stream is not closed or destroyed, which leads to a memory leak.

✅So if you use pipeline, it would close all other streams and make sure that there are no memory leaks.

const { createReadStream } = require('fs');
const { createServer } = require('http');
const { pipeline } = require('stream');

const server = createServer(
  (req, res) => {
    pipeline(
      createReadStream(__filename),
      res,
      err => {
        if (err)
          console.error('Pipeline failed.', err);
        else
          console.log('Pipeline succeeded.');
      }
    );
  }
);

server.listen(3000);

Grooved

The documentation also states that this almost exact example prohibits you from interacting with res. So you can't send back a 500 status since when createReadStream closes, it will close res as well. Then it calls the callback. That doesn't mean it's "bad". Just beware. – Assimilative

Answer 2

According to the documentation, they both do the same thing. But there some differences:

• .pipe() is a method of Readable, while pipeline is a module method of stream that accepts streams to pipe.
• pipeline() method provide a callback when the pipeline is complete.
• pipeline() method was added since 10 version of node, while .pipe exist from the earliest versions of Node.

In my opinion, with pipeline() code looks a bit cleaner, but you can use both of them.

Example of .pipe():

const fs = require('fs');
const r = fs.createReadStream('file.txt');
const z = zlib.createGzip();
const w = fs.createWriteStream('file.txt.gz');
r.pipe(z).pipe(w);

Example of pipeline():

const { pipeline } = require('stream');
const fs = require('fs');
const zlib = require('zlib');

pipeline(
  fs.createReadStream('archive.tar'),
  zlib.createGzip(),
  fs.createWriteStream('archive.tar.gz'),
  (err) => {
    if (err) {
      console.error('Pipeline failed.', err);
    } else {
      console.log('Pipeline succeeded.');
    }
  }
);

Narcoma

Answer 3

pipeline is the improved version of pipe it was added to stream module since Node.js v10

Also, pipeline takes any number of arguments and the last argument is a callback used to know when the pipeline ends or throws an error.

Usage pipe:

mySourceStream.pipe(myStream).pipe(anotherStream)

Usage pipeline:

mySourceStream.pipeline(myStream, anotherStream, err => {
  if (err) {
    console.log('There is an error')
  } else {
    console.log('pipeline successful')
  }
})

Inconsumable