Correct way to implement HTTP Connection Pooling

Question

I am using Apache HTTP Client for connection pooling during my REST API calls into certain web services.

Strange thing is that in spite of me using HTTP Connection Pooling there are no gain in my performance.

I am using Apache HTTP Client to connect to my web services, and the code is as follows from there documentation :

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();

cm.setMaxTotal(200);

cm.setDefaultMaxPerRoute(20);

HttpHost host = new HttpHost("abc.com", 80);
cm.setMaxPerRoute(new HttpRoute(host), 50);

CloseableHttpClient httpClient = HttpClients.custom()
        .setConnectionManager(cm)
        .build();

I am using Spring's RestTemplate to wrap around the HttpClient implemenation of Apache using Spring's HttpComponentsClientHttpRequestFactory.

But even if I use no connection pooling ie. use the SimpleClientHttpRequestFactory of Spring, I get no performance advantage.

My connections still take the same amount of time to complete.

Is what I have done the correct way to implement HTTP Connection Pooling? Am I doing something wrong?

Please let me know if any further info is required from my side.

Allyson

I am of the assumption that PoolingHttpClientConnectionManager is used for multi-threading case, as it will help multiple connections concurrently. I might be wrong here, can someone please explain this correctly? – Addieaddiego

Answer 1

Beware of how HTTP Client pools work, it may be improving performance during a short period of time. Check the analysis below:

From PoolingHttpClientConnectionManager javadocs

The handling of stale connections was changed in version 4.4. Previously, the code would check every connection by default before re-using it. The code now only checks the connection if the elapsed time since the last use of the connection exceeds the timeout that has been set. The default timeout is set to 2000ms

From the pool performance perspective it means that a connection to a particular route will be reused as long as the manager considers that route as "active" during a period of 2 seconds by default.
After 2 seconds of inactivity connections to that route will be considered stale and discarded thus incurring in a connection init penalty next time that route is requested.

In other words, out of the box, the pool improves performance for connections after the first during 2 seconds. Heavy duty routes are the most benefited.

As a simple test, set your pool size to an small value, like 5 max. Send 5 requests and check the number of established connections to that route, on linux

watch "ss -ant dst <your route IP>"

watch "netstat -ant | grep <your route IP>"

You should see 5 connections. Wait 10 or 20 seconds and send 2 requests to the same route, you should see those 5 connections closed and 2 new created. It's also possible to observe that with debug logging. Here is a good article as reference.
Apache official docs on http logging.

Umlaut

Answer 2

I believe you are setting up the HttpClient and pooling manager correctly.

My implementation has one slight difference in that instead of using HttpClients.custom(), I am using HttpClientBuilder.create(), but then have the same method invocations as you have. According to this other Stack Overflow answer, this shouldn't make a difference.

I've used this configuration before in Spring applications and had good benefits. I wonder if maybe the response is happening fast enough that you don't see a large benefit? The only other thing I can think of is if potentially the RestTemplate isn't getting configured correctly.

Superfamily

AFAIK the RestTemplate is just a wrapper around HTTPClient. – Allyson

Effectively that is correct. That was just the only code I didn't see in your original question :). Luis Muñoz's response below seems like a good candidate for why you aren't seeing the performance gains you thought you would. – Superfamily

Answer 3

Your configuration seems to be correct. You can use multi-threading to use system's resources to gain performance.

    HttpGet get = new HttpGet("http://www.codersjargon.com");

    PoolingHttpClientConnectionManager connManager 
      = new PoolingHttpClientConnectionManager();

    CloseableHttpClient client = HttpClients.custom().
        setConnectionManager(connManager).build();

     MultiHttpClientConnThread thread1 = new MultiHttpClientConnThread(client, get);
     MultiHttpClientConnThread thread2 = new MultiHttpClientConnThread(client, get);
     MultiHttpClientConnThread thread3 = new MultiHttpClientConnThread(client, get); 
     thread1.start();
     thread2.start();
     thread3.start(); 
     thread1.join(); 
     thread2.join();
     thread3.join();

Zoroaster

I dont want multi threading since the results of the previous calls are needed for the subsequent calls. – Allyson

@Zoroaster Can you help me understand if PoolingHttpClientConnectionManager is advantageous only in case of multi-threading? – Addieaddiego