How to get "requests per second" for Apache in Linux?

Question

In Windows for ASP, you can get it perfmon, but...

How to get "requests per second" for Apache in Linux?

Handyman

The question is not clear. Can you specify is you ask for realtime or historical logs? – Paginate

Answer 1

In realtime, or can you use mod_status?

And apparently, there is a version of top for apache...

Fussy

The mentioned link was dead on 2009 based on web.archive.org, but it had addressed to this project: github.com/nexcess/apachetop – Kaye

Answer 2

Here is a short bash script I made up to sample the request rate (based on dicroce's suggestion of using wc -l on the log file).

#!/bin/sh

##############################################################################
# This script will monitor the number of lines in a log file to determine the
# number of requests per second.
#
# Example usage:
# reqs-per-sec -f 15 -i /var/www/http/access.log
#
# Author: Adam Franco
# Date: 2009-12-11
# License: http://www.gnu.org/copyleft/gpl.html GNU General Public License (GPL)
##############################################################################

usage="Usage: `basename $0` -f <frequency in seconds, min 1, default 60> -l <log file>"

# Set up options
while getopts ":l:f:" options; do
 case $options in
 l ) logFile=$OPTARG;;
 f ) frequency=$OPTARG;;
 \? ) echo -e $usage
  exit 1;;
 * ) echo -e $usage
  exit 1;;

 esac
done

# Test for logFile
if [  ! -n "$logFile" ]
then
 echo -e $usage
 exit 1
fi

# Test for frequency
if [  ! -n "$frequency" ]
then
 frequency=60
fi

# Test that frequency is an integer
if [  $frequency -eq $frequency 2> /dev/null ]
then
 :
else
 echo -e $usage
 exit 3
fi

# Test that frequency is an integer
if [  $frequency -lt 1 ]
then
 echo -e $usage
 exit 3
fi

if [ ! -e "$logFile" ]
then
 echo "$logFile does not exist."
 echo 
 echo -e $usage
 exit 2
fi

lastCount=`wc -l $logFile | sed 's/\([0-9]*\).*/\1/'`
while true
do
 newCount=`wc -l $logFile | sed 's/\([0-9]*\).*/\1/'`
 diff=$(( newCount - lastCount ))
 rate=$(echo "$diff / $frequency" |bc -l)
 echo $rate
 lastCount=$newCount
 sleep $frequency
done

Maxma

Great! Thanks! I am wondering how the performance of wc -l is. Another approach would be the possibility to pipe the logfile to stdin of the script. So also grep could be added to meassure only .php requests and so on. – Moa

In the comments section, the option to specify the log file should be -l instead of -i – Hypersonic

Answer 3

To sum up, you can use mod_status and apachetop.

Alternatively, you can use Adam Franco's and Jon Daniel's nice scripts to have a live look.

If you would like to have a look at a partiular date and hour, you can issue this little command:

grep "29/Oct/2014:12" /var/log/apache2/example.com.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Replace with the date and hour you are interested and also with the proper pathfilename of the log file.

It will print out something like:

1913 12:47
 226 12:48
 554 12:49
 918 12:50

There is a nice article here with more options on using a combination of awk, cut and uniq commands to get quick stats of the kind.

Photoelectric

Answer 4

You can use 'wc -l' on the access log to get the number of lines (which roughly corresponds to the number of requests...) Do that every minute and subtract the last value to get the delta...

Flitter

Answer 5

I didn't like any of the solutions I found, so I wrote my own.

• mod_status isn't really accurate enough. It's based on how long the server is up, which in our case is normally months. What I'm looking for is traffic spikes.
• the shell script above uses a sleep() statement which isn't ideal as it takes x seconds to actually retrieve the data.

So this solution takes a particular line in the access_log 15000 requests ago, and uses the time recorded to compare with the current time.

# This check is needed because if the logs have just rolled over, then we need a minimum
# amount of data to report on.
# You will probably need to adjust the 3500000 - this is roughly the file size when the
# log file hits 15000 requests.
FILESIZE=`ls -l /var/log/httpd/access_log | awk '{print $5}' `
if [ $FILESIZE -le 3500000 ]
then
        # not enough data - log file has rolled over
        echo "APACHE_RPS|0"
else
        # Based on 15000 requests.  Depending on the location of the date field in
        # your apache log file you may need to adjust the ...substr($5... bit
        LASTTIME=`tail -15000 /var/log/httpd/access_log | head -1 | awk '{printf("%s\n",substr($5,2,20));}' `
        APACHE_RPS=`echo $LASTTIME | gawk -vREQUESTS=15000 ' {
                # convert apache datestring into time format accepted by mktime();
                monthstr = substr($0,4,3);
                if(monthstr == "Jan"){ monthint = "01"; }
                if(monthstr == "Feb"){ monthint = "02"; }
                if(monthstr == "Mar"){ monthint = "03"; }
                if(monthstr == "Apr"){ monthint = "04"; }
                if(monthstr == "May"){ monthint = "05"; }
                if(monthstr == "Jun"){ monthint = "06"; }
                if(monthstr == "Jul"){ monthint = "07"; }
                if(monthstr == "Aug"){ monthint = "08"; }
                if(monthstr == "Sep"){ monthint = "09"; }
                if(monthstr == "Oct"){ monthint = "10"; }
                if(monthstr == "Nov"){ monthint = "11"; }
                if(monthstr == "Dec"){ monthint = "12"; }
                mktimeformat=sprintf("%s %s %s %s %s %s [DST]\n", substr($0,8,4), monthint, substr($0,1,2), substr($0, 13,2), substr($0, 16,2), substr($0, 19,2) );
                # calculate difference
                difference = systime() - mktime(mktimeformat);
                # printf("%s - %s = %s\n",systime(), mktime(mktimeformat), difference);
                printf("%s\n",REQUESTS/difference);
        } ' `

        echo "APACHE_RPS|${APACHE_RPS}"
fi

Prestigious

Answer 6

I think mod_status can do it ...

http://httpd.apache.org/docs/2.0/mod/mod_status.html

You can also use zenoss to collect data from mod_status using the community apache plugin.

http://www.zenoss.com/

Ignite

Answer 7

Script shows inconsistent numbers. -f parameter affects output a lot! and first reading is not accurate either.

I ended up using:

while true; do tail -n0 -f access.log>/tmp/tmp.log & sleep 2; kill $! ; wc -l /tmp/tmp.log | cut -c-2; done 2>/dev/null

Found here.

Tired

what does the cut -c-2 do? isn't that just cutting the first two characters in the line? That seems wrong. I am getting numbers like 500+ and cut gives me 11. – Nutriment

Answer 8

I wrote a set of Perl scripts that show the average requests-per-second for the past 1, 5 and 15 minutes (like top). It's at https://gist.github.com/1040144 .

Wacke

Answer 9

mod_status is the one! if you call it with:

http://{ip}/server-status?refresh=1&auto-refresh=true

Then it auto refreshes every 2 seconds so you can see a constant realtime view :-)

Impostume

Answer 10

Should someone prefer to get a list of the number of requests per second for each second in the Apache access log for a particular API endpoint (as opposed to the number of requests per minute, as provided in the @Wtower's answer), here is a charming one-liner:

grep "api/v1" /var/log/apache2/access.log | \
    cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3":"$4}' | awk -F' ' '{print $1}' | \
    sort -t':' -nk1 -nk2 -nk3 | uniq -c | awk '{ if ($1 > 1) print $0}'

Stearin