Could not get a resource from the pool(SocketTimeoutException:)

Question

I'm running multiple worker threads(around 10) to access the data from the redis Q.
For the i'm using infinte timeout for Jedis Client.

Jedis jedis = pool.getResource();
jedis.getClient().setTimeoutInfinite();  

Still i'm getting the error "Could not get a resource from the pool". The stacktrace is given below.

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:22)
at Workers.Worker1.met1(Worker1.java:124)
at Workers.Worker1.work(Worker1.java:108)
at org.gearman.impl.worker.WorkerConnectionController$3.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)  

Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
at redis.clients.jedis.Connection.connect(Connection.java:124)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:54)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1657)
at redis.clients.jedis.JedisPool$JedisFactory.makeObject(JedisPool.java:63)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1188)
at redis.clients.util.Pool.getResource(Pool.java:20)
... 6 more  

Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at redis.clients.jedis.Connection.connect(Connection.java:119)
... 11 more

Antiphon

And is your Redis server alive? Can you connect to it from the client box using redis-cli? – Yecies

Yes Redis server is alive and able to connect using redis-cli – Antiphon

I have the same problem. Redis is running. The problem occurs when using JedisPool, and I do returnResource. No problem with Jedis jedis = new Jedis("localhost");. Have you solved this issue? – Dixon

in my case it was from very small timeout in (millisecond) – Baudin

Answer 1

I noticed that this exception can and will be thrown if Redis is not running. Just a heads up.

Prisage

Answer 2

Based on Rick Hanlon's answer, this exception is also thrown if using Redis with Spring Boot.

If you're using Spring Boot, just the dependency of Redis is not enough; you also need to manually download and install Redis on your machine from redis.io, then run it off of a Bash terminal:

me@my_pc:/path/to/redis/dir$ ./src/redis-server ./redis.conf

After running the server, you'll need to add the relevant lines in all your apps that use Redis:

application.properties:

...
spring.redis.host: <yourhost> // usually localhost, but can also be on a LAN
spring.redis.port: <yourport> // usually 6379, but settable in redis.conf

application.yml:

...
spring:
  redis:
    host: <yourhost> // usually localhost, but can also be on a LAN
    port: <yourport> // usually 6379, but settable in redis.conf

Markusmarl

Is redis installation needed, even when connecting to remote redis instance? – Rocambole

@Rocambole no it is not – Jenny

Thanks figured that out later by reading more docs. It was a stupid question though :) – Rocambole

Answer 3

Did it happen always or occasionally? If it was occasional, you may want to check the size of your connection pool.

Default connection pool size, if you are using JedisPoolConfig, is 8. This could be too small for your case.

JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(128);
jedisPool = new JedisPool(poolConfig, HOST, PORT, ...);

Punch

Could you pls guide me #51409469 ? – Mitochondrion

Answer 4

Possible causes;

1 - Redis server is down or Redis application not responding.

2 - Application can not connect to Redis server (firewall etc. issues).

3 - Connection to Redis server timed out.

4 - All connections in the (Redis) pool are currently busy, new connection can not be allocated.

The cases 1 and 2 are infra related.

For case 3, connection timeout must be increased ("RedisConnectionTimeout"):

pool = new pool(poolConfig, RedisIp, RedisPort, RedisConnectionTimeout);

For case 4, max connection count must be increased ("RedisMaximumActiveConnectionCount"):

poolConfig.setMaxTotal(RedisMaximumActiveConnectionCount); 

Assuming the following or similar implementation;

private Pool<Jedis> pool =  null;   

private final String RedisIp="10.10.10.11";
private final int RedisPort=6379;
private final String RedisConnectionTimeout=2000;
private final String RedisMaximumWaitTime=1000;
private final String RedisMaximumIdleConnectionCount=20;
private final String RedisMaximumActiveConnectionCount=300;
private final String SentinelActive=false;
private final String SentinelHostList="10.10.10.10:26379,10.10.10.10:26380,10.10.10.10:26381";
private final String SentinelMasterName="sentinel-master-name";

private synchronized void initializePool()
{
    if(pool!=null) return;

    poolConfig poolConfig = new poolConfig();
    poolConfig.setMaxTotal(RedisMaximumActiveConnectionCount); 
    poolConfig.setMaxIdle(RedisMaximumIdleConnectionCount);  
    poolConfig.setMaxWaitMillis(RedisMaximumWaitTime); 

    if(SentinelActive)
    {
        String [] sentinelsArray = SentinelHostList.split(",");

        Set<String> sentinels = new HashSet<>();            
        for(String sentinel : sentinelsArray)
        {
            sentinels.add(sentinel);
        }

        String masterName = SentinelMasterName;

        pool = new JedisSentinelPool(masterName, sentinels, poolConfig, RedisConnectionTimeout);            
    }
    else
    {       
        pool = new pool(poolConfig, RedisIp, RedisPort, RedisConnectionTimeout);
    }

}           

protected Jedis getConnection()
{               
    if(pool==null)
        initializePool();

      Jedis jedis = pool.getResource();

      return jedis;     
}   

Build

Answer 5

If your code is like this:

JedisPoolConfig jedisPoolConfig = initPoolConfig();    
jedisPool = new JedisPool(jedisPoolConfig, "*.*.*.*", 6379);  

You can try this:

JedisPoolConfig jedisPoolConfig = initPoolConfig();    
jedisPool = new JedisPool(jedisPoolConfig, "*.*.*.*", 6379,10*1000); 

This is because for Redis the default timeout time is 2 seconds, but the program may have finished running in this time.

Cascade

Answer 6

Not sure, but maybe you don't return Jedis objects to the pool, and your redis-server has connection limit.

Every worker thread should return Jedis instances to the pool after its work is complete:

Jedis jedis = jedisPool.getResource();
try {
    jedis.getClient().setTimeoutInfinite();
    // your code here...
    ...
} finally {
    jedisPool.returnResource(jedis);
}

Novelize

yes @Jarek, already i'm returning the resource. pool.returnResource(jedis); pool.destroy(); – Antiphon

In my Java App I'm using Jedis2.0.0.jar file and redis server version is 2.4.13. Is it making the problem? – Antiphon

returnResource is protected method, you cannot use it. You need to use try with resources try (Jedis jedis = pool.getResource()) {} – Farrier

Answer 7

On AWS Elastic Cache (Redis) if

Redis AUTH default user access is Yes, you need to set SSL=true in JedisPool like this

boolaen SSL=true;
jedisPool = new JedisPool(poolConfig, redisServer, redisServerPort, redisTimeout, redisPassword, SSL);

Hyetograph

what should be the password ? – Faints

Answer 8

I've gotten that error when jedis simply could not reach the desired redis instance (domain/ip, port, or password), which could happen if they are either wrong, or they were right before and now are wrong, or if they are right but a firewall is blocking access to them, etc.

I know the OP confirmed having run redis-cli to confirm the connection, but it would have been helpful to confirm if that was done from the same place the jedis call was coming from (which could have been an app server leveraging jedis to connect to some redis server).

But that original post was from 2012, so I don't expect to hear an update but I leave these for others who may find this thread.

Degauss

Answer 9

I had to struggle a lot to solve the 'Could not get a resource from the pool' error but I finally solved it and hopefully it will work out for you as well!

Change port in 'application.properties' from '6380' to '6379' It should work fine then!

Agamemnon

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Mcdonnell

Simplified the post! – Agamemnon

Answer 10

I just want to add one point in @mrt's and @charlie's answers. We need to check the properties of redis server. For redis client 3.2.0 and above we can set ssl properties as well, with the help of JedisShardInfo as follows:

JedisShardInfo shardInfo = new JedisShardInfo(redisServer, redisServerPort, redisTimeout, redisSsl);
shardInfo.setPassword(redisPassword);
Jedis jedis = new Jedis(shardInfo);

and with jedisPool:

jedisPool = new JedisPool(poolConfig, redisServer, redisServerPort, redisTimeout, redisPassword, redisSsl);
jedis = jedisPool.getResource();

Where, in poolConfig, you can specify various properties like, setMaxIdle, setMaxWaitMillis, setMaxTotal and so on.

Eductive

Answer 11

This happenned to my application, actually I was using a single Object which was being called by multiple functions and those were not thread safe.

Problem was multiple calls on a single object leading to corrupted calls.

Something like this :

Class A{
    Object B;
    function1(){
        B.doSomething();
    }
    function2(){
        B.doSomething();
    }
}

As they were not threadsafe, I was getting these errors :

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Socket is closed

and

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out

This is how I fixed it :

Class A{
    function1(){
        Object B;
        B.doSomething();
    }
    function2(){    
        Object B;
        B.doSomething();
    }
}

Hope it helps

Frederickfredericka