Scoreboard is full,not at MaxRequestWorkers

Question

I an using Apache and Tomcat with mod-jk in my project. My web application is in Tomcat instances. I am using Apache jmeter for testing http load on tomcat. But I'm getting the error Scoreboard is full,not at MaxRequestWorkers even with 1000 threads.

[mpm_event:error] [pid 24313:tid 3075319488] AH00485: scoreboard is full, not at MaxRequestWorkers

[mpm_event:error] [pid 24313:tid 3075319488] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

I've configured http-mpm for approximately 5000 threads.

httd-mpm.conf file is:

<IfModule mpm_event_module>
    ServerLimit              200
    StartServers             3
    MinSpareThreads          75
    MaxSpareThreads          250
    MaxClients               5000 
    ThreadsPerChild          25
    MaxRequestWorkers        5000
    MaxConnectionsPerChild   5000
</IfModule>

Can anybody please guide me what can be the problem in this configuration?

Smew

bz.apache.org/bugzilla/show_bug.cgi?id=53555 – Albite

I concur that the above Bug is observed as a possible issue. (bz.apache.org/bugzilla/show_bug.cgi?id=53555 ). I also observed that the server will correct itself after a while and kill child processes (e.g. [Mon Nov 16 15:09:27.456764 2015] [core:error] [pid 1793:tid 139889295517568] AH00046: child process 8375 still did not exit, sending a SIGKILL – Mortar

Answer 1

It seems that Apache tries to gracefully finish some of the processes and hangs while doing so. It might be a problem of Apache (https://bz.apache.org/bugzilla/show_bug.cgi?id=53555) or a problem of the application itself.

You can easily check it by looking at the server-status page of apache. In order to do that you need to enable the status.conf apache module. Then go to your-server.org/server-status. You will see all the processes and threads currently being used or gracefully finishing ("G"-state). If you have to many G's that don't go away, you found your problem. It would look sth like this:

Too many threads gracefully finishing...

The solution is pretty simple. Just set MaxConnectionsPerChild to 0 or comment it out (mpm_event.conf). What this does is just let the process continue it's work without restarting it. That's a good thing in our case, since restarting the process let to the problem of having to many G's filling up the scoreboard and eventually crashing Apache.

Hope that helped.

Saimon

This was a livesaver - completely fixed the problem on Apache 2.4.10, which comes with Debian (Jessie) 8.9. Thanks so much! – Karlkarla

Still an issue with Apache 2.4.57. Setting MaxConnectionsPerChild 0 doesn't fix it for me. – Kaka

Answer 2

I had this same problem. I tried different Apache versions and MPMs.

I seem to get this alot with MPM Worker. Also error does not reoccur using Apache 2.2.2

Are you using cPanel? IF so try /upcp --force and increase StartServers to a higher amount like 50 as that's all I did to get this error away.

Halfprice

Answer 3

This serverfault Q&A suggests that the event mpm may have a bug. The recommended settings for at least partial workaround:

StartServers 3
MinSpareServers 5
MaxSpareServers 10
ServerLimit 250
MaxRequestWorkers 250
MaxConnectionsPerChild 1000
KeepAlive Off

Mortification

Answer 4

Try EnableMMAP Off in 00_default_settings.conf

Downstate