Single-instance Redis lock with SETNX

Question

I need to connect to a single Redis instance from an application client.

Since the client will be replicated in Kubernetes, I'm studying Redis documentation about locks to prevent races between the client replicas.

After some googling and reading around, I zeroed in on these two resources:

• the SETNX command described here: https://redis.io/commands/setnx
• the Redlock algorithm described here: https://redis.io/topics/distlock

Interestingly the SETNX docs explicitly advise against using SETNX to implement locks, stating that it has basically become obsolete:

The following pattern is discouraged in favor of the Redlock algorithm [...]

We document the old pattern anyway because certain existing implementations link to this page as a reference.

However the Redlock algorithm is specifically tailored for distributed locks, thus when one seeks to lock on multiple Redis instances - they actually refer to multiple masters.

To go a bit further, the library redsync (golang) declares the New function as follows:

func New(pools []Pool) *Redsync {
    return &Redsync{
        pools: pools,
    }
}

It looks unmistakably designed to support locking on a Redis cluster.

In my use case, I'm going to connect to only one Redis instance.

Probably I can just use the redsync package and pass an slice of length one, yet to me it looks like the SETNX pattern could work equally fine on a single Redis instance.

Am I seeing this correctly? Thank you

Pyknic

Answer 1

Yes, it's true that the Redlock algorithm is designed for a distributed Redis system, and that if you're using a single instance it should be fine to use the simpler locking methods described in the SET and SETNX documentation.

However, a more important point is this: you probably don't need to use locks to avoid conflicts between multiple Redis clients. Redis locks are usually used to secure some external distributed resource (see my answer here for a bit more on this). Within Redis itself locks are generally not necessary; thanks to Redis' single-threaded nature, many commands are already atomic, and you have the ability to use transactions or Lua scripts to compose arbitrarily complex atomic operations.

So my advice is to deisgn your client code to use atomicity to avoid conflicts rather than trying to use a lock (distributed or otherwise).

Deadly

Answer 2

Figured I could post an answer for reference. Following Kevin's advice, I ended up using a Lua script to ensure atomicity.

This is how the implementation (Go) looks like:

// import "github.com/gomodule/redigo/redis"

type redisService struct {
    pool      *redis.Pool
    lastLogin *redis.Script // Lua script initialized into this field
}

// Constructing a redis client
func NewRedisService(config *config.RedisClientConfig) RedisService {
    return &redisService{
        pool: &redis.Pool{
            MaxIdle:     10,
            IdleTimeout: 120 * time.Second,
            Dial: func() (redis.Conn, error) {
                return redis.Dial("tcp", config.BaseURL)
            },
            TestOnBorrow: func(c redis.Conn, t time.Time) error {
                if time.Since(t) < time.Minute {
                    return nil
                }
                _, err := c.Do("PING")
                return err
            },
        },
        // initialize Lua script object
        // lastLoginLuaScript is a Go const with the script literal
        lastLogin: redis.NewScript(1, lastLoginLuaScript),
    }
}

Lua script (the comment explains what it does):

--[[
    Check if key exists, if it exists, update the value without changing the remaining TTL.
    If it doesn't exist, create it.

    Script params
    KEYS[1] = the account id used as key
    ARGV[1] = the key TTL in seconds
    ARGV[2] = the value
]]--
local errorKeyExpired = 'KEXP'
local statusKeyUpdated = 'KUPD'
local statusKeyCreated = 'KCRE'

if redis.call('exists', KEYS[1]) == 1 then
    local ttl = redis.call('ttl', KEYS[1])
    if ttl < 0 then --[[ no expiry ]]--
        redis.call('setex', KEYS[1], ARGV[1], ARGV[2])
        return redis.status_reply(statusKeyCreated)
    end
    if ttl == 0 then --[[ expired ]]--
        return redis.error_reply(errorKeyExpired)
    end

    redis.call('setex', KEYS[1], ttl, ARGV[2])
    return redis.status_reply(statusKeyUpdated)

else
    redis.call('setex', KEYS[1], ARGV[1], ARGV[2])
    return redis.status_reply(statusKeyCreated)
end

Usage:

func (rs *redisService) UpsertLastLoginTime(key string, ttl uint, value int64) (bool, error) {
    conn := rs.pool.Get()
    defer conn.Close()

    // call Do on the script object
    resp, err := rs.lastLogin.Do(conn, key, ttl, value)

    switch resp {
    case statusKeyCreated:
        return true, nil

    case statusKeyUpdated:
        return false, nil

    case errorKeyExpired:
        return false, ErrKeyExpired

    default:
        return false, errors.Wrap(err, "script execution error")
    }
}

Pyknic