Redis streams in cluster mode

Question

Do Redis streams benefit from Cluster mode? Imagine you had 10 streams, would they be distributed across the cluster or all on the same node? I'm planning on using Redis streams for really high throughput (2m+ messages/s) so I'm worried about the performance of Redis streams at this scale.

Any guidance towards scaling Redis streams horizontally would be awesome if it doesn't scale out of the box in Cluster mode.

Drisko

Answer 1

would they be distributed across the cluster or all on the same node? I

It depends on the keys of these streams. Redis Cluster distributes these streams based on the key of each stream. If these keys have the same hash tag, they will be located on the same node. Otherwise, the it takes the CRC16 of the key to distribute the streams. Check this for detail.

Uhl

This effectively means streams do not scale with clusters. However different stream could slot themselves into different shards based on stream name - so that is one knob available for tuning the distribution, – Pneuma