What is the benefit of Async Servlets

Question

I was reading up on Async servlets 3.1

It is said that the thread that sends the request is freed up when there is lot of time consuming operations to be done for e.g. fetching long list of data from the database.

I am not able to understand the benefit here because anyways there will be a new thread allocated to process the database connection and response processing even if the initial thread that was responsible for the request is freed.

So how is Async servlet more beneficial than thread per request model that we had earlier.

Imperfection

Answer 1

Answering in two parts here:

1. How is Async servlet more beneficial than thread per request model that we had earlier: This model is long dead, and almost all Java servers use NIO, which allows these servers to handle hundreds of connections using a handful of threads. You might verify this for your app server as well, and you will be pleasantly surprised to see that it does use NIO :). The Async Servlets do not have anything to do with the one request per thread.
2. Then why Async Servlets: Well, Async Servlets allow for the original request to be completed, without waiting for the async task completion(which hopefully is long running). Thus the remote client can be responded immediately, and they can do some other stuff if they have to. The async task can be processed by the server later in one of the threads. These Async Operations are typically done in a separate thread pool, meant for Async operations. The thread pool meant for handling client connections is not used for Async operations.

Update: Some more details on why we need Async Servlets if we were already using NIO thread pool I did jot down my notes sometime ago on non-blocking IO at http://manish-m.com/?p=996 . You might also view a related post at http://manish-m.com/?p=915 (particularly the IO Playground section on this page).

The NIO thread pool is for handling multiple connection requests. It uses non-blocking IO feature of kernels, so that a small number of threads can work with many connections.

However, the same threads that read data from the network buffer also executes the "user code" (that we write within servlets). The servlet container's framework for NIO handles accepting client request, but it cannot handle a "blocking user code" on its own, that is written by us. Thus, if we write a DB Query that takes say 10 seconds, then the containers framework cannot handle it asynchronously by itself. We would block the original NIO thread pool by writing any blocking code in servlets. Hence, we need to explicitly write anything, that we think can potentially block the request threads of container, as a Async servlet in Java EE.

Similarly, when we use other NIO frameworks like Netty, MINA, then we need to take care of ensuring that the code "does not" block the NIO threads that handle the network connections. This is usually achieved by off-loading such long running tasks to another thread pool (which is what the container does when you write an async servlet).

Leotaleotard

WRT to #1 above I think you are confusing what threads are in play with how NIO works. Even servers which implement their selector functionality with NIO can and will still implement a thread pool to handle a thread per request handed from the application server to the application (eg. an Async servlet). For example see this explanation of Grizzly's NIO connector architecture jfarcand.wordpress.com/2006/01/26/… – Fiden

@Fiden Not sure what portion of the article you are referring to. What I read in Grizzly architecture page is - "This strategy prevent one thread per request, and enable Grizzly to server more that 10 000 concurrent users with only 30 threads". Can you point to the specific section in the article? – Leotaleotard

Grizzly is handling the low level IO using NIO. This layer is independent of the upper layers like the servlet container which an Async servlet would be a part of. In between you have a request thread pool, managed as a container resource, which is 1 thread per request. Its these threads which the Async servlet allows to be freed back up while its long running request is serviced thus allowing the web app to better scale. – Fiden

@Fiden Agree, and that is what I was trying to articulate in #2 of my answer "These Async Operations are typically done in a separate thread pool, meant for Async operations." :) However, only the async operations use a separate thread pool. Other, non-async servlets, run in the basic NIO thread pool, and you could exhaust them easily by performing "blocking" operations in the servlet code (by not using the Async Servlet API). – Leotaleotard

For #1, it only applies for static files, comet and web socket. For static files the web server can use sendfile with MMAP/DMA, zero copy! For comet and web socket applications, the programming model is async and well mapped to NIO APIs, that's cool! For servlet, it's a totally different story. e,g. Tomcat, only the headers are parsed before your servlet by the endpoint processor, but the post body is under control of your servlet, getInputStream is bridged by the underlying InternalNioInputBuffer, although it's NIO but it's still blocking, just to satisfy servlet spec. – Slashing

Answer 2

There are only so many threads available to service HTTP requests. These come from the HTTP thread pool and configurable via your application servers configuration mechanism (eg. Glassfish's administration UI). If your servlet performs long running operations these request threads will be occupied. Once all pooled request threads are occupied no further requests will able to be serviced until one is freed up.

With an Async servlet the request thread will be returned to the pool as opposed to being blocked by the long running synchronous operation in a non-async situation.

Fiden

But isnt the Async operation like for e.g. Data Retrieval from DB going to consume a new thread to release the one to service HTTP Requests – Imperfection

Yes it will use a thread for the Async operation. Chances are the thread it uses will also be managed and pooled, for example via a Managed Executor resource or pooled EJB. The benefit of Async Servlet is to improve the scalability of the web application with regards to idle container request threads. – Fiden

@kapilchhattani But isnt the Async operation like for e.g. Data Retrieval from DB going to consume a new thread - Yes, that's a real risk. If you're not careful you'll just be swapping out one thread for the other. If both the HTTP threads and async processing threads come from a pool (which is typically the case) you could just have well assigned more threads to the HTTP pool, since the net effect will be nothing. – Deppy

@ArjanTijms you are totally correct! There is no sense (or at least I haven't found it yet) in using AsyncServlet model if you are working with embedded servers, because you can always adjust the total HTTP threads pool. But this is completely a different story if you run your application in a provided servlet container which you cannot configure. In this case, the AsyncModel is super helpful. – Strutting