Async Spring Controllers vs normal Controllers

Question

I wanted to analyze the improvement I may see by enabling Async Controllers in Spring Boot over normal controller

So here is my test code. One API returns a Callable and another is normal controller API. Both APIs block for 10secs simulating a long running task

@RequestMapping(value="/api/1",method=RequestMethod.GET)
    public List<String> questions() throws InterruptedException{
        Thread.sleep(10000);
        return Arrays.asList("Question1","Question2");
    }

    @RequestMapping(value="/api/2",method=RequestMethod.GET)
    public Callable<List<String>> questionsAsync(){
        return () -> {
            Thread.sleep(10000);
            return Arrays.asList("Question2","Question2");
        };
    }

I set up embedded tomcat with this configuration i.e only one tomcat processing thread:

server.tomcat.max-threads=1
logging.level.org.springframework=debug

Expectations for /api/1 Since there is only one tomcat thread, another request will not be entertained untill this is processed after 10secs

Results: Meet expectations

---

Expectations for /api/2 Since we are returning a callable immediately, the single tomcat thread should get free to process another request. Callable would internally start a new thread. So if you hit the same api it should also gets accepted.

Results: This is not happening and untill the callable executes completely, no further request is entertained.

Question Why is /api/2 not behaving as expected?

Semiautomatic

Tomcat is running a threadpool, you got wrong expectations. – Unbutton

@RomanC I mentioned in question, i set the tomcat's threadpool to contain only 1 thread. – Semiautomatic

Just to be sure: which kind of "other request" do you submit while the spring thread is sleeping? – Hondo

@JBNizet other request is the same api request and its not spring thread, it actually tomcat request processing thread that my question is about.(nio-8082-exec-1) – Semiautomatic

Try to add logging and have a look at the threadIds to figure out if the callable is executed in a different thread. Did you configure a TaskExecutor? Spring MVC recommends to do so otherwise the SimpleAsyncTaskExecutor is used. – Fresh

You didn't actualy say how you tested your assertions. It works as expected for me (if I have 4 concurrent consumers I see roughly 4x more throughput with the async controller). I measured it with Apache bench running on a different machine than the server. – Hemihedral

@DaveSyer Thanks. I tested with curl and I can see /api/2 behave as expected. As mentioned below, its infact an issue with chrome browser – Semiautomatic

Answer 1

@DaveSyer is right, /api/2 is actually behaving as expected.

I assume you are testing the behavior with a web browser. At least Firefox and Chrome are preventing multiple simultaneous requests to the same URL. If you open 2 tabs with api/2, the second one will only send a request to the application after the first got the response.

Try testing it with a simple bash script, like:

curl localhost/api/2 &
curl localhost/api/2 &
curl localhost/api/2 &

It will print 3 responses around the same time.

Martyrdom

You and @DaveSyer are absolutely right! This issue is happening only in browser and not with curl or any http client test tool. – Semiautomatic

Answer 2

Just want to mention that server.tomcat.max-threads is deprecated since Spring boot 2.3. Now use server.tomcat.threads.max in your Spring application.properties. The default is 200.

Rochet