pipelining vs transaction in redis

Question

When we use a transaction in Redis, it basically pipelines all the commands within the transaction. And when EXEC is fired, then all the commands are executed together, thus always maintaining the atomicity of multiple commands.

Isn't this same as pipelining?

How are pipelining and transaction different? Also, why does not the single threaded nature of Redis suffice? Why do we explicitly need pipelining/transaction?

Wholly

Answer 1

Pipelining is primarily a network optimization. It essentially means the client buffers up a bunch of commands and ships them to the server in one go. The commands are not guaranteed to be executed in a transaction. The benefit here is saving network round trip time for every command.

Redis is single threaded so an individual command is always atomic, but two given commands from different clients can execute in sequence, alternating between them for example.

Multi/exec, however, ensures no other clients are executing commands in between the commands in the multi/exec sequence.

Correlation

for clarification suppose if I use jedis as following : JEDIS.MULTI \\ line 1; JEDIS.command1 \\ line 2; JEDIS.command2 \\ line 3; JEDIS.EXEC \\ line 4; then for each line this code will go to the redis server ,thus four rounds back and forth from redis server. However, if I pipeline the transaction , then all the four lines will go to the redis server in one round.Is that right? – Wholly

@ManasSaxena, I believe not. I think the client will buffer the commands and send all at once on the call to exec. You can test this yourself with the redis-cli. Open 2 terminals with redis-cli. On the first, run MONITOR. On the second run the following commands GET a, MULTI, GET b, GET c, EXEC. You will see that GET a (which is outside of the transaction) is logged on the MONITOR right away, while GET b and GET c only are logged when EXEC is executed. Not 100% sure, but is my current understanding. The buffering could happen server side, but I don't see a reason. – Demoniac

I did an extra digging and found something contrary what I said in the previous comment. If I stop my Redis server after GET b, GET c will fail. So it seems that the redis-cli tries to communicate with the server on every command. But this seems to be something that each client can choose. I've read that clients can send transactions all at once, queuing them as pipelines on the client side. Like this: redislabs.com/ebook/part-2-core-concepts/…. But they could just add a simple phrase explaining this – Demoniac

@ManasSaxena Ok, after some digging, the answer to your question is: it depends on the implementation of Jedis. Jedis could open a transaction and send the commands one by one (I confirmed that the commands are queued on the server in this issue: github.com/antirez/redis-doc/issues/1203#issuecomment-547475496). But it also could send all the commands at once, at the exec. So to be sure, you should open Jedis code and check. I honestly tried it, but is full of classes hierarchy that made me give up. – Demoniac

Unfortunately, MULTI and EXEC aren’t free, and can delay other important commands from executing Non-transactional pipelines, so transaction can delay other important commands, it's not atomic and not guaranteed. – Felic

you should also be aware that redis does not have any transaction rollback like you'd get in a relational database's transactions (#39586403) – Carman