A race condition when using Redis command incr and expire

Question

Based on the redis document: http://redis.io/commands/incr

In the paragraph Pattern: Rate Limiter 2 A shorter version code:

value = INCR(ip)

IF value == 1 THEN
  EXPIRE(ip, 1)

It's claimed there's a race condition to make EXPIRE never executes. Which means the value of ip can bounces from 0 to 2 some way.

However in my thoughts, since Redis is single thread and INCR is a primitive command, it shouldn't be atomic itself? Even if 2 clients do the INCR at almost the same time, how could them both retrieve 0 or both retrieve 2?

Loyola

See also What is the race condition for Redis INCR Rate Limiter 2 – Anterior

Answer 1

Imagine that you drop connection to redis server after INCR command was already executed but before EXPIRE was executed. In this case you never execute EXPIRE becouse as next call of code gives your value > 1. In redis documentation the term race condition used. But it is not successful term. A more correct term is imperfect algorithm. So this case not about race condition between 2 or more clients but about special cases in real world. Server connection loss for example.

Staal

I am also doing incr and then expire ... is there any way to prevent this? like atomic incr and expire... is lua script the only option? – Farlay

Answer 2

It is still possible to achieve what you want in a atomic way: you can use the EVAL command.

EVAL is used to execute a script written in Lua inside the Redis server, the best part of it is that this script is executed like a single atomic operation.

The following script can be used with this purpose:

local v = redis.call('INCR', ARGV[1]) if v == 1 then redis.call('EXPIRE', ARGV[1], ARGV[2]) end return v

The logic is really simple: We are storing the return value of a INCR command into a variable labeled v, then we check if v value is 1 (first increment) if it is, we call the command EXPIRE for that key and then we return the value of v. The ARGV[...] are the parameters passed to the script, ARGV[1] is the key name and ARGV[2] is the timeout in seconds for the given key.

Example using this script:

> eval "local v = redis.call('INCR', ARGV[1]) if v == 1 then redis.call('EXPIRE', ARGV[1], ARGV[2]) end return v" 0 my_key 10

(integer) 1

> eval "local v = redis.call('INCR', ARGV[1]) if v == 1 then redis.call('EXPIRE', ARGV[1], ARGV[2]) end return v" 0 my_key 10

(integer) 2

> get my_key

"2"

[wait 10 seconds]

> get my_key

(nil)

Landmark

I knew this solution, simply curious about the original question I posted. Thanks anyway :) – Loyola

Answer 3

I met the same question, what about this : value = INCR(ip) IF [ value == 1 || PTTL(ip) == -1 ]THEN EXPIRE(ip, 1)

If 2 clients do the PTTL at almost the same time, get -1 ,and set expire at almost same time

Jowers