Jersey Client non-blocking

Question

Spawning lots of threads is never a good idea (and when you create too many, you may run out of memory anyway).

Usually, Jersey needs to create one thread per request. And this seems to be the case, whether I use async() ^{(where Jersey creates the threads for me - I've investigated this in a debugger)}, or not ^{(where I obviously have to create the threads myself)}.

So here's one concrete situation where this is not good enough:

I'm HTTP posting to remote servers, at a rate of up to 500 requests/second. But as the response may take some time to arrive (I calculate up to 30 seconds), the total number of threads can reach several thousands easily (at which point, a JVM process usually crashes). Moreover, it's just insane to create so many threads. It should actually be a piece of cake for the available processor/network/OS-resources to deal with that load.

So what I'd like to do, is to just fire off the requests - and be informed by the OS, when the HTTP response arrives.

• As said above, simply using target.request(...).async()....doesn't do the trick (because then, Jersey just spawns its own threads).
• Also, limiting the number of threads via new ClientConfig().property(ClientProperties.ASYNC_THREADPOOL_SIZE, 10) is not helpful at all, because it means, that at most 10 requests will be sent at a time, which is clearly not what I want (it would just pile-up the queue).

I experimented with new ClientConfig().connectorProvider(new GrizzlyConnectorProvider())to get NIO support - but didn't see any difference in behaviour at all.

So is there any way to fire off a request without having to create one extra thread per request?

Rutger

I just found, that there is jersey-non-blocking-client, and a corresponding blog post, where the author addresses the problem (though he seriously underestimates the impact of non-blocking vs blocking). However, unfortunately the project is for an old version of Jersey, which I cannot use. Maybe newer versions of Jersey already come with non-blocking support? And if yes, how to activate it? – Rutger

The number of threads you can start is firstly a matter of heap space. There is nothing magic about 2048. – Assisi

@EJP: Thanks, I corrected the question. – Rutger

@EJP: BTW, the 2048 limit I see seems to be enforced by Mac OS X: sysctl kern.num_taskthreads shows kern.num_taskthreads: 2048 - and it seems, that the Oracle JVM actually uses kernel threads (interesting). – Rutger

Answer 1

I am using the CloseableHttpAsyncClient to make async requests to an external service. it works fine with few hundred requests per second, i haven't observed that number of threads like in your case. It's an external dependency which you can integrate through Maven via

<dependency>
  <groupId>org.apache.httpcomponents</groupId>
  <artifactId>httpasyncclient</artifactId>
  <version>4.0.1</version>
</dependency>

Hope this helps.

Indurate

In the end, I went with Apache's HttpAsyncClient (I also tried AsyncHttpClient as suggested by @Alper, which also works great - just little differences). As far as I can see, the current versions of Jersey cannot deliver nonblocking behavior, because it is limited by JAX-RS 2.0 (a few places mention, that this might become possible with JAX-RS 3.0) – Rutger

Answer 2

Make your requests using https://github.com/AsyncHttpClient/async-http-client, it uses netty. It'll fire it's calls off the request thread, and make a call back to your code, so it does not tie up the container request threads.

Rosemaria

(I deleted my previous comment, I had made a mistake) Yes, AsyncHttpClient does it right - it creates a worker thread pool, and can deal with much more active requests than workers. However, it would be quite a lot of effort to rework our application to use AsyncHttpClient, as we use a lot of Jersey features... wouldn't it be possible to achieve the same with Jersey? – Rutger

We use Jersey for our service endpoints, and when we require a call to another service, we use AsyncHttpClient. If you are already using the jersey client, I guess that may not be super easy to change. You can use an ExecutorService and a future, to make the call off the calling thread and accomplish the same thing. I'll add another example. – Rosemaria

Using my own ThreadPool doesn't really solve the problem: If the ThreadPool is bounded, and I use sync calls to Jersey, then the callers to the executor will block. If I use async calls to Jersey, then Jersey unfortunately creates its own threads (not just workers, but one per open request)... (the only thing that could maybe work, would be to use sync calls, and then kill my thread - but that feels like a very bad idea.) – Rutger

You're right, that was not going to solve anything, I removed that example. – Rosemaria

You've take a look at this I guess as well, as a possibility? – Rosemaria

@AlperAkture's suggestion of using Jersey async with AsyncHttpClient is money. Exactly what I was looking for. – Lateshalatest