How to implement transaction with rollback in Redis

Question

My program needs to add data to two lists in Redis as a transaction. Data should be consistent in both lists. If there is an exception or system failure and thus program only added data to one list, system should be able to recover and rollback. But based on Redis doc, it doesn't support rollback. How can I implement this? The language I use is Java.

Chaiken

Answer 1

If you need transaction rollback, I recommend using something other than Redis. Redis transactions are not the same as for other datastores. Even Multi/Exec doesn't work for what you want - first because there is no rollback. If you want rollback you will have to pull down both lists so you can restore - and hope that between our error condition and the "rollback" no other client also modified either of the lists. Doing this in a sane and reliable way is not trivial, nor simple. It would also probably not be a good question for SO as it would be very broad, and not Redis specific.

Now as to why EXEC doesn't do what one might think. In your proposed scenario MULTI/EXEC only handles the cases of:

1. You set up WATCHes to ensure no other changes happened
2. Your client dies before issuing EXEC
3. Redis is out of memory

It is entirely possible to get errors as a result of issuing the EXEC command. When you issue EXEC, Redis will execute all commands in the queue and return a list of errors. It will not provide the case of the add-to-list-1 working and add-to-list-2 failing. You would still have your two lists out of sync. When you issue, say an LPUSH after issuing MULTI, you will always get back an OK unless you:

• a) previously added a watch and something in that list changed or
• b) Redis returns an OOM condition in response to a queued push command

DISCARD does not work like some might think. DISCARD is used instead of EXEC, not as a rollback mechanism. Once you issue EXEC your transaction is completed. Redis does not have any rollback mechanism at all - that isn't what Redis' transaction are about.

The key to understanding what Redis calls transactions is to realize they are essentially a command queue at the client connection level. They are not a database state machine.

Angieangil

Interesting, if redis can't rollback, why does it claims that its transaction is atomic? – Instinct

Answer 2

Redis transactions are different. It guarantees two things.

1. All or none of the commands are executed
2. sequential and uninterrupted commands

Having said that, if you have the control over your code and know when the system failure would happen (some sort of catching the exception) you can achieve your requirement in this way.

1. MULTI -> Start transaction
2. LPUSH queue1 1 -> pushing in queue 1
3. LPUSH queue2 1 -> pushing in queue 2
4. EXEC/DISCARD

In the 4th step do EXEC if there is no error, if you encounter an error or exception and you wanna rollback do DISCARD.

Hope it makes sense.

Athene

Answer 3

There is a way, how clever/wrong it is, is up to you to decide.

As other colleagues have noticed, Redis transactions guarantee atomicity, but not at all consistency. Also, everything between MULTI/EXEC|DISCARD block is sent as a single command, so if you're using any kind of higher-level Redis client library, you practically have no control of what's going on along the way.

If there's a real need to tamper with intermediary results or perform conditionals based on them or else - the default choice would be to write some Lua scripts. Inside the script you can basically do anything, limited only by Lua capabilities itself and some Redis environment specifics.

Scripts inside the transaction would be executed one by one, and you cannot stop it. However, you can skip specific scripts or even parts of them.

In order to do that, each transaction scripts batch must be provided with a unique ID (UUID or whatever), passed as an argument (let's call it trx_id):

--[[
  Minor advice: define keys/args tables with local aliases upfront,
  makes things a lot easier while working with the script itself.
--]]

local keys = {
  ...
}

local args = {
  ...
  trx_id = ARGV[13]
}

Now, if some condition is unmet and you want to stop the subsequent scripts execution, set a unique lock with the trx_id end exit:

--[[
  Minor advice: using hash tag in lock key name will make it drop into the same hash
  slot thus playing nice in the clustering mode (refer to the docs for more info).
--]]

local placeholder = 1

if (true) then
  if args.trx_id then redis.call('SET', '{halt_mark}:' .. args.trx_id, placeholder, 'PX', 1000, 'NX') end
  error(...)
end

Without mentioning some sophisticated edge cases, this will set up a lock. Now, you have an anchor which can be used by subsequent scripts in the batch given that they've been provided with the same trx_id:

--[[
  Minor advice: args must be always checked for existence, because Redis
  will silently allow to call a script with less arguments than expected.
--]]

local keys = {
  ...
}

local args = {
  ...
  trx_id = ARGV[42]
}

if args.trx_id and not redis.call('SET', '{halt_mark}:' .. args.trx_id, 1, 'PX', 3000, 'NX') then
  error(...)
end

This simple condition will try to obtain the lock and fail if it's already taken, meaning that one of the previous operations decided that it would be enough.

In the essence, it allows you to skip next scripts in a transaction based on the "halt mark" which is set when you need to kill the transaction mid-way.

Don't forget to release the lock at the end of an each script in the chain, if everything went well:

if args.trx_id then redis.call('DEL', '{halt_mark}:' .. args.trx_id) end

As I've meant from the beginning, this is not a recommended way / best practice in any case, but it will mimic the relational DB style transactions in terms of consistency (especially if provided with tailored rollback actions on errors) when there's no other way.

Putout