Apache threads stay in state reading after queries

Question

My configuration is apache and tomcat behind an aws elb. Apache is configured with no keepalive, and set with max clients to a low number due to each query being very cpu intensive. I'll load test the machine with queries. Then the number of available requests goes to zero as can be seen by curl -s localhost/server-status?auto not responding immediately. When I stop the loadtest I can see the scoreboard from curl -s localhost/server-status?auto still is full of R's even though from the tomcat logs it is clear nothing is happening. Does anyone have an idea on what possible causes there might be?

Assegai

Have you tried testing without the ELB in the middle? That way you can confirm whether it's Apache or the ELB which is is holding onto connections. Likewise have you checked headers to confirm that Keep-Alive is not being sent because this changes the ELB's behaviour. – Mayman

Answer 1

If your apache display 'R' in the status it means there is open TCP connections from ELB to apache (just an open TCP connection, no data sent yet).

There is no official complete documentation on this subject (how the numbers of pre-opened connections is optimized ), but the amazon documentation state (at this page: https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/how-elastic-load-balancing-works.html ) that:

Classic Load Balancers use pre-open connections but Application Load Balancers do not.

So, the answer is: it is an optimization from amazon (TCP connection cost a little bit to be openned).

Incomprehensible