Parallel service calls with Spring Boot 3.2 and virtual threads
Asked Answered
D

2

6

I am trying to call two external services in parallel using virtual threads on spring boot 3.2 with RestClient, but I'm not sure if it is correct to simply call them sequentially or use futures. My application.properties already contains: spring.threads.virtual.enabled=true

Is it correct to simply do:

String response1 = performGetRequest(uri1);  //RestClient get request
String response2 = performGetRequest(uri2);

return response1 + response2;

Or do I need to do:

Future<String> response1 = performGetRequest(uri1);
Future<String> response2 = performGetRequest(uri2);

return response1.get() + response2.get();

Also, is it necessary to wrap the block of code in a try-with-resources block, or is there no need because spring is already using virtual thread per task as it is enabled in application.properties? As so:

try (ExecutorService service = Executors.newVirtualThreadPerTaskExecutor();)
Dappled answered 17/2, 2024 at 16:33 Comment(0)
A
4

If you can use virtual threads with your applications, prefer using the latter approach i.e. invoke both the REST API calls using newVirtualThreadPerTaskExecutor. A similar use case for fanout scenarios is even suggested in the documentation of virtual threads under the section #Represent Every Concurrent Task as a Virtual Thread; Never Pool Virtual Threads.

First Approach

Spring by itself would make use of the virtual threads to take care of the requests initiated to the server. This would mostly mean that every request would be assigned to a new virtual thread handled by Tomcat and these threads depending on the I/O performed within each request can switch to accepting further requests considering higher throughput and concurrency.

But once a thread is assigned the request (consider tracing a request-id), it will carry out all subsequent synchronous steps until it responds within the same thread. If logs could help you relate, this would result in an output like the following:

GreetingsController       : 34ace5cc-2b6b-4226-a088-b5888fc30f99 : VirtualThread[#568,tomcat-handler-51]/runnable@ForkJoinPool-1-worker-10
WebPageService            : 34ace5cc-2b6b-4226-a088-b5888fc30f99 : tomcat-handler-51 : /uri1
WebPageService            : 34ace5cc-2b6b-4226-a088-b5888fc30f99 : tomcat-handler-51 : /uri2

Second Approach

With the approach of using newVirtualThreadPerTaskExecutor, you are assigning all the service calls to a new virtual thread as a task concurrently. In this case, the application further spawns new virtual threads from the common ForkJoinPool for each of your service calls submitted. Here are the logs for reference for this case:

GreetingsController       : ad39b86e-073c-479e-a5d8-ec35bbb34dac : VirtualThread[#575,tomcat-handler-54]/runnable@ForkJoinPool-1-worker-12
WebPageService            : ad39b86e-073c-479e-a5d8-ec35bbb34dac : VirtualThread[#576]/runnable@ForkJoinPool-1-worker-12: /uri1
WebPageService            : ad39b86e-073c-479e-a5d8-ec35bbb34dac : VirtualThread[#580]/runnable@ForkJoinPool-1-worker-12 : /uri2

Side Note - The thread name in the above logs is added with the help of a debugger.


Additionally, for your question about the usage of try-with-resources, quoting the referenced link summarising the usefulness of the newly introduced ExecutorService.close:

The close method, that is implicitly called at the end of the try block will automatically wait for all tasks submitted to the ExecutorService—that is, all virtual threads spawned by the ExecutorService—to terminate.

This could certainly help release the resources for the other threads to make use of and reduce any possible leaks in the application.

Airbrush answered 17/2, 2024 at 19:58 Comment(2)
Thank you for your input. So basically it is ideal to enable virtual threads in application properties AND on top of that assign every task / call within the service to a newVirtualThreadPerTaskExecutor try-with-resources block so that every other task is running on virtual threads as well. In addition to that, for fanout scenarios Futures should still be used to call services in parallel, but written in the imperative style, as there is no benefit from using asynchronous style coding with CompletableFuture (besides chaining operations).Dappled
@WilliamN True. Besides, writing the Futures with a submit as compared to the asynchronous CompleteableFuture depends further on the use cases and the outcome as desired.Airbrush
V
1

Once Spring is concerned, there is more Spring-native and elegant way to do the parallel calls than to use Java barebone Executors.newVirtualThreadPerTaskExecutor. If your method performGetRequest is advisable method, i.e. declared in a Spring bean and public, then you could annotate it with @Async, and then your second way

Future<String> response1 = performGetRequest(uri1);
Future<String> response2 = performGetRequest(uri2);
return response1.get() + response2.get(); 

will execute in parallel.

For example,

@Service
public class GetRequestPerformer {

    @Async
    public Future<String> performGetRequest(String url) {
        ... invoke RestClient
        return CompletableFuture.completedFuture(result);
    }
}

Note that @Async should be properly configured:

@Configuration
@EnableAsync
public class AppConfig {
}

and a call to performGetRequest should be advised (proxied): an instance of GetRequestPerformer should be injected:

@Autowired
private GetRequestPerformer performer;

Under the hood, Spring (by default) uses SimpleAsyncTaskExecutor, which implements Spring's TaskExecutor and does not have all rich capabilities of ExecutorService (they are unlikely needed in your context), but still implements AutoClosable, has a lifecycle of Spring Application Context and will attempt to gracefully close all running threads when the context closes.

If you prefer bareboning, a completely barebone way (which in fact demonstrates a simplicity of parallelism with virtual threads), is:

final String[] results = new String[2];
Thread t1 = Thread.ofVirtual().start(() -> results[0] = performGetRequest(url1));
Thread t2 = Thread.ofVirtual().start(() -> results[1] = performGetRequest(url2));
t1.join();
t2.join();
return results[0] + results[1];

and then performGetRequest method needs not to be annotated with @Async and returns plain String instead of Future. This approach does not need any Spring/Spring Boot support and a configuration of spring.threads.virtual.enabled property.

Vaccinate answered 28/2, 2024 at 10:49 Comment(3)
Thank you for your input! But the question is if @Async is using SimpleAsyncTaskExecutor under the hood, isn't this using platform threads instead of virtual threads? Should the SImpleAsyncTaskExecutor be configured to use Virtual Threads?Dappled
Edit: Replying to my comment, I just read that spring boot auto configures the async task executor to use virtual threads when spring.threads.virtual.enabled=true. Thanks : )Dappled
Yes, exactly. In fact, org.springframework.boot.autoconfigure.thread.Threading is set to VIRTUAL when Spring Environment contains spring.threads.virtual.enabled=true. Then all other supporters of VTs like task executors, async handlers, Kafka, Redis etc reuse this value.Vaccinate

© 2022 - 2025 — McMap. All rights reserved.