NodeJS: What's the difference between a Duplex stream and a Transform stream?

Question

The Stream docs state that Duplex Streams "are streams that implement both the Readable and Writable interfaces" and Transform Streams "are Duplex streams where the output is in some way computed from the input." Unfortunately, the docs do not describe what Transform streams provide above and beyond Duplex streams.

Are there any differences between the two? When would you use one over the other?

Crowley

Answer 1

A Duplex stream can be thought of a readable stream with a writable stream. Both are independent and each have separate internal buffer. The reads and writes events happen independently.

                             Duplex Stream
                          ------------------|
                    Read  <-----               External Source
            You           ------------------|   
                    Write ----->               External Sink
                          ------------------|
            You don't get what you write. It is sent to another source.

A Transform stream is a duplex where the read and write takes place in a causal way. The end points of the duplex stream are linked via some transform. Read requires write to have occurred.

                                 Transform Stream
                           --------------|--------------
            You     Write  ---->                   ---->  Read  You
                           --------------|--------------
            You write something, it is transformed, then you read something.

Episiotomy

I think I understand! Transform Streams must implement the _transform method, which has the same signature as a Writeable Stream's _write method, so this implies that what's read depends on what's written. One implication of this is that you can't specify the number of bytes to be read at a time like with _read, it's simply the number of bytes you wrote. On the other hand, with Duplex streams, you could theoretically make a _read which depends on _write, or even have the _write which depend on _read, but no read/write linking is implied by the Duplex Stream class itself. – Crowley

+1 for the ascii diagram, they should be used in the node.js docs that would help convey the concepts much better – Plymouth

upvoted so by your definition a single transform stream should only generate a single transformed output, so lets say I have 1m ohlc data coming live through a stream and I want to generate 5m 15m etc from it, I would only do 1 conversion per instance of transform stream or would it be considered good practice to push all the timeframes from a single input? – Perithecium

Aren't the two types of streams implemented in the same way though, and just presented to us developers with different methods? (write and read for duplex, transform for transform streams) – Shamrao

Answer 2

The difference is just syntactic sugar. Transform streams are full duplex streams but rather than implementing _write and _read methods, you are asked to implement just the _transform method. You can read more about streams on the excellent substack's streams guide or from Isaacs's readable-stream repo.

Langobardic

Answer 3

According to the docs:

Duplex - streams that are both Readable and Writable (for example, net.Socket).

Transform - Duplex streams that can modify or transform the data as it is written and read (for example, zlib.createDeflate()).

So to put it simply:

• A duplex stream can be read from and written to, but there's not necessarily any connection between the input and output.
• A transform stream can also be read from and written to, but the output will always be the result of a transformation on its input. Transform streams are Duplex streams where the output is in some way related to the input.

Shamrao

Answer 4

If you read the API for stream implementors section of the docs, they state that some possible use cases for Duplex and Transform streams are "Reading and writing" and "Operate on written data, then read the result" respectively.

Simply put the Transform stream lets you implement the _transform method that takes some input and returns the output after some operation has been done on the data, and can be used for things like compressing or hashing, whereas the Duplex stream can be used for things like a TCP socket connection where you just send and receive data.

Vergos