Too many RedisCommandTimeoutException in lettuce

Question

We are facing this specific issue using lettuce redis library. We are receiving too many RedisCommandTimeoutException. We have set a timeout of 2 secs in redis-cli and 10 ms in redis slow logs. While nothing gets logged in slowlogs our application keeps getting this timeout.

Code we are using is as follows

Duration timeout = 
Duration.ofMillis(applicationProperties.redisTimeOut);
RedisClient client = RedisClient.create(RedisURI.create(applicationProperties.redisUrl));
client.setDefaultTimeout(timeout);
RedisCommands<String, String> commands = client.connect().sync();

We have about 100 threads in our application wgich might be using this shared connection

Exception we receive is as follows

io.lettuce.core.RedisCommandTimeoutException: Command timed out
at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:114)
at io.lettuce.core.FutureSyncInvocationHandler.handleInvocation(FutureSyncInvocationHandler.java:62)
at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)
at com.sun.proxy.$Proxy11.hmget(Unknown Source)

Least

Did you manage to figure this out? Having the same problem at the moment. – Dichromaticism

we have got a lead though not 100% sure if that is the only cause . Incase of network fluctuation it takes a good amount of time for library to recover and hence in most cases throws timeout exception. A good idea would be to use a lower value for timeout and implement retries – Least

Yeah that's what I was trying to avoid having to do. I did manage to run some load testing of about 100 million requests to our Redis Cluster using lettuce with the connection pool. Have not had a single command timeout since. Have you considered trying that? – Dichromaticism

hey can you let me know about this in details as I believe that connection pooling will ensure that subsequent request don't die but how does it help to the request already initiated and impacted by network fluctuation. – Least

I have such a problem. how do you think if the problem is the Redis itself? I mean there may be misconfigurations which makes the Redis not to be able to handle a large number of simultaneous requests. – Sollows

any solution you found? I am facing the same problem on my localhost, so no reason to consider network legging. – Uranian

Anyone figure this out? Happening to my prod environment and can’t figure out what’s causing the timeouts. – Puglia

Answer 1

I had the same error and it was that redis fell suddenly and when doing the query it took a long time and several retries, I found this way, I think there is a better way, but it worked for me

@Bean
public LettuceConnectionFactory redisConnectionFactory() {
        RedisStandaloneConfiguration redisConf = new RedisStandaloneConfiguration();
        redisConf.setHostName(env.getProperty("spring.redis.host"));
        redisConf.setPort(Integer.parseInt(env.getProperty("spring.redis.port")));
        redisConf.setPassword(RedisPassword.of(env.getProperty("spring.redis.password")));

        LettuceConnectionFactory factory = new LettuceConnectionFactory(redisConf);
        factory.setTimeout(500L); //timeout to redis

        return factory;
    }

Bipartite

You basically increased the timeout, didn't your application became slow? And was redis went down in your case? I have these exceptions too but Redis wasn't stressed at all. So In my case I think I need to look at connection pooling. Still reading a bit on this. – Balmuth