Does Reactive programming consumes more resources than non-reactive?

Question

We are currently facing a performance issue with spring webFlux. For the sake of ascertaining the benefits of the reactive programming we have implemented a Spring Boot service which fetches data from a MongoDB and returns it via a REST API to the consumers.

The service exists in two variants:

1. A non-reactive implementation with Spring Boot, MongoRepository. This service returns the data as List
2. A reactive implementation with Spring Boot, ReactiveMongoRepository, spring-boot-starter-webflux. This service returns the data as Flux.

In both implementations the REST controller directly fetches the data from the repository and returns it as List resp. as Flux. No further application logic is executed.

We conducted a small load/performance test with 100 users calling the service and we found out that the non-reactive implementation performed far better than the reactive implementation.

As a matter of fact, not only had the non-reactive implementation a better HTTP throughput but, perhaps more interestingly, it consumed less CPU and less threads than the reactive implementation! This was particularly counter to expectations since we anticipated the reactive version to scale with a small number of thread as mentioned in https://spring.io/blog/2016/07/28/reactive-programming-with-spring-5-0-m1

Is there something we need to tweak in the settings?

Has someone faced a similar issue?

Channel

Answer 1

We performed a similar test using Spring-Data-Reactive-Cassandra and Spring-Webflux against Spring-Data-Cassandra and Spring-MVC.

We benchmarked both server for 10000 request with a concurrency of 100 req/sec. The results weren't much surprising:-

Non Reactive Stack:-
         Concurrency Level:      100
         Time taken for tests:   22.945 seconds
         Complete requests:      10000
         Failed requests:        0
         Percentage of the requests served within a certain time (ms)
           50%    190
           66%    253
           75%    288
           80%    314
           90%    384
           95%    465
           98%    627
           99%    824
          100%   1208 (longest request)

Reactive Stack:-
         Concurrency Level:      100
         Time taken for tests:   30.061 seconds
         Complete requests:      10000
         Failed requests:        0
         Percentage of the requests served within a certain time (ms)
           50%    304
           66%    379
           75%    421
           80%    443
           90%    507
           95%    589
           98%    694
           99%    736
          100%    858 (longest request)

While performing these tests, the non-reactive stack spawned 147 threads, while the reactive stack spawned 48 threads.

If you compare the results, the non reactive stack is slightly faster as compared to the reactive stack. It persisted 10,000 objects in the database in roughly 23 seconds while reactive stack took roughly 30 seconds. However if you compare the slowest 2% requests in the two stacks the reactive stack is almost 28% faster.

Reactive stack, with a lesser number of threads had a more evenly distributed response times. None of the requests were stalled. While for the non reactive stack the 1% of request were terribly slow comparatively.

On increasing the number of calls over a continued period of time, reactive stack will be able to scale much better as compared to non-reactive stack. Since the number of threads you can spawn on the server is much less as compared to the number of sockets you can open on the server. Also During these tests the CPU utilisation was less than 33% in both the scenarios, thereby proving CPU utilisation was not restricting scalability.

The non-reactive stack could scale more if it hadn't been restricted by Thread-Context switching and Thread creation.

Mountaintop

Answer 2

Let me try to explain the possible reasons for this behavior. A reactive application does not work faster than a reactive application. A reactive application does not allow the system to stand idle if the request queue is not empty. Since you have tested at low load, you have not seen the pros and cons of a reactive application, but you have seen downgrade performance. The performance is lower than that of an inactive application, because the reactive execution has a small overhead.

Yippie