Using connection pool with JSch

Question

I'm using JSch for file upload over SFTP. In its current state each thread opens and closes connection when needed.

If it possible to use connection pooling with JSch in order to avoid overhead caused by large number of connection opening and closing?

Here is a example of function called from inside of thread

 public static void file_upload(String filename) throws IOException {
    JSch jsch = new JSch();
    Session session = null;
    try {
        session = jsch.getSession("user", "server_name", 22);
        session.setConfig("StrictHostKeyChecking", "no");
        session.setPassword("super_secre_password");
        session.connect();

        Channel channel = session.openChannel("sftp");
        channel.connect();
        ChannelSftp sftpChannel = (ChannelSftp) channel;
             
        FileInputStream inputSrr = new FileInputStream(filename);  
        try {  
            sftpChannel.put(inputSrr, "/var/temp/"+filename);  
        } catch (SftpException e) {  
            e.printStackTrace();  

        } finally {  
            if (inputSrr != null) {  
                inputSrr.close();  
            }  
        }  
        
        sftpChannel.exit();
        session.disconnect();
    } catch (JSchException e) {
        e.printStackTrace();
    } catch (SftpException e) {
        e.printStackTrace();
    }
}

Chamorro

Answer 1

For that I would prefer commons-pool. ;)

Epigastrium

Was it helpful to you? I'm asking you, because stackoverflow is built on the reputation score system. – Epigastrium

I was busy with other project so didn't have the time to try it out. I'll let you know as soon as I give it a go. – Chamorro

Can you share how you implemented it? – Korfonta

@Korfonta I have added simple implementation, if you need more details feel free to ask me. I would add more info how to cofigure pool and how to use it – Pollard

Answer 2

I wold like to share with you our implementation, We have used Session Manager of jsch-extension library

First of all you need to implement pool object factory that is responsible for lifecycle of pooled objects:

public class ChannelSftpConnectionsFactory extends BasePooledObjectFactory<ChannelSftp> {
    private SessionManager sessionManager;

    public ChannelSftpConnectionsFactory(final SessionManager sessionManager) {
        this.sessionManager = sessionManager;
    }

    //Create and open channel
    @Override
    public ChannelSftp create() throws JSchException {
        ChannelSftp channelSftp = (ChannelSftp) sessionManager.getSession().openChannel("sftp");
        channelSftp.connect();

        return channelSftp;
    }

    //wrapping
    @Override
    public PooledObject<ChannelSftp> wrap(final ChannelSftp channelSftp) {
        return new DefaultPooledObject<>(channelSftp);
    }

    @Override
    //disconnect channel on destroy
    public void destroyObject(final PooledObject<ChannelSftp> pooledObject) {
        ChannelSftp sftp = pooledObject.getObject();
        disconnectChannel(sftp);
    }

    void disconnectChannel(final ChannelSftp sftp) {
        if (sftp.isConnected()) {
            sftp.disconnect();
        }
    }

    @Override
    //reset channel current folder to home if someone was walking on another folders
    public void passivateObject(final PooledObject<ChannelSftp> p) {
        ChannelSftp sftp = p.getObject();
        try {
            sftp.cd(sftp.getHome());
        } catch (SftpException ex) {
            log.error("Could not reset channel to home folder, closing it");
            disconnectChannel(sftp);
        }
    }

    @Override
    //validate object before it is borrowed from pool. If false object will be removed from pool
    public boolean validateObject(final PooledObject<ChannelSftp> p) {
        ChannelSftp sftp = p.getObject();
        return sftp.isConnected() && !sftp.isClosed();
    }
}

Now you could create pool using configured factory:

ObjectPool<ChannelSftp> createPool(final SessionManager sessionManager, final GenericObjectPoolConfig<ChannelSftp> poolConfig) {
    return PoolUtils.synchronizedPool(new GenericObjectPool<>(buildFactory(sessionManager), poolConfig));
}


PooledObjectFactory<ChannelSftp> buildFactory(final SessionManager sessionManager) {
    return PoolUtils.synchronizedPooledFactory(new ChannelSftpConnectionsFactory(sessionManager));
}

This java doc would help you to configure pool properly : https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/impl/BaseGenericObjectPool.html

Do not forget about correct borrowing and returning of object into pool: https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/ObjectPool.html

 Object obj = null;

 try {
     obj = pool.borrowObject();
     try {
         //...use the object...
     } catch(Exception e) {
         // invalidate the object
         pool.invalidateObject(obj);
         // do not return the object to the pool twice
         obj = null;
     } finally {
         // make sure the object is returned to the pool
         if(null != obj) {
             pool.returnObject(obj);
        }
     }
 } catch(Exception e) {
       // failed to borrow an object
 }

Pollard

Answer 3

Here's an implementation of Ssh Connection pool http://www.javacodegeeks.com/2013/02/pool-of-ssh-connections-using-apache-keyedobjectpool.html

you can use grep4j to use this pool https://code.google.com/p/grep4j/source/browse/trunk/src/main/java/org/grep4j/core/command/linux/SessionFactory.java?r=354

Also make sure you can access the server from the execution machine. For instance if the target server is not in your reach. It'll throw connection timeout.

Hairdresser

The implementation will cause classpath issues on newer versions of java as grep4j is no longer maintained – Agan