How to force Apache Bench to use IP addresses specified in /etc/hosts?

Question

I'm using ApacheBench to do some load testing. I'd like ab to resolve the hostname using the IP address specified in /etc/hosts on my Mac. How can I force that? curl has a --resolve option to do exactly this, as specified here. I'm looking for something similar for ab.

Klansman

Answer 1

Think of it the other way around: You can tell Curl to hit an IP address, and specify the Host header to trigger the desired domain.

curl example.com
curl --header "Host: example.com" 93.184.216.34

The same method works with ab using the -H flag, as such

ab -n 10 -c 10 http://example.com/
ab -n 10 -c 10 -H "Host: example.com" http://93.184.216.34/

Hope this helps.

Edit:

Piggybacking off @Magistar's answer, you want to ensure you are using the correct protocol (http vs https) and are hitting the correct FQD. That is to say, if your website responds to www.example.com but not example.com (without the www) be sure to use the one that works.

Another thing to note is make sure you are not load testing a redirect. For example, running ab against yahoo.com (I do not suggest doing this) will not be a good test, as yahoo.com responds with an immediate redirect to www.yahoo.com which you can see if you curl it in verbose mode:

# curl yahoo.com
redirect
# curl yahoo.com -v
* About to connect() to yahoo.com port 80 (#0)
*   Trying 98.139.180.180...
* Connected to yahoo.com (98.139.180.180) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: yahoo.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Wed, 07 Mar 2018 13:46:53 GMT
< Connection: keep-alive
< Via: http/1.1 media-router-fp29.prod.media.bf1.yahoo.com (ApacheTrafficServer [c s f ])
< Server: ATS
< Cache-Control: no-store, no-cache
< Content-Type: text/html
< Content-Language: en
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=2592000
< Location: https://www.yahoo.com/
< Content-Length: 8
<
* Connection #0 to host yahoo.com left intact
redirect

Or more specifically, since this is about spoofing a host to a destination IP address:

# curl --header "Host: yahoo.com" 98.139.180.180 -v
* About to connect() to 98.139.180.180 port 80 (#0)
*   Trying 98.139.180.180...
* Connected to 98.139.180.180 (98.139.180.180) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> Host: yahoo.com
>
< HTTP/1.1 301 Moved Permanently
< Date: Wed, 07 Mar 2018 13:51:26 GMT
< Connection: keep-alive
< Via: http/1.1 media-router-fp82.prod.media.bf1.yahoo.com (ApacheTrafficServer [c s f ])
< Server: ATS
< Cache-Control: no-store, no-cache
< Content-Type: text/html
< Content-Language: en
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=2592000
< Location: https://www.yahoo.com/
< Content-Length: 8
<
* Connection #0 to host 98.139.180.180 left intact
redirect

So be sure to curl the url first, to ensure it is returning the data you expect, and then proceed with ab.

Phanerogam

Thanks, this works! (I had omitted the trailing / in the IP address so it gave me an error yelling "invalid URL" so it took a while for me to figure out what the error was.) – Klansman

what if i add manual entry in /etc/hosts :-) – Liquidator

@Liquidator - That could also work, but is not an option without sudo, or if you are nesting multiple end-points (load balancers, dev envs, etc). – Phanerogam

Answer 2

For SSL based websites to prevent getting 0 bytes:

ab -n 10 -c 10 -H "Host: www.example.com" https://93.184.216.34/

The trick is to include the subdomain (www) to the host name.

(ps didn't have enough points to just add it as comment).

Drolet

Thanks, this is a useful comment. – Klansman

I must note that when I recently compared my ab results with 'real' results there was a difference. With real results I mean the time to first byte as displayed in your browser (f12>network tools). For some websites AB was able to reach 250ms where the browser would display 900ms. So this method still isn't perfect. The value of AB did scale nicely with more cpu power though. So it is not useless. – Drolet