I tried to benchmark Scotty to test the Network I/O efficiency and overall throughput.
For this I set up two local servers written in Haskell. One which doesn't do anything and just acts as an API.
Code for the same is
{-# LANGUAGE OverloadedStrings #-}
import Web.Scotty
import Network.Wai.Middleware.RequestLogger
import Control.Monad
import Data.Text
import Control.Monad.Trans
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network
main = do
scotty 4001 $ do
middleware logStdoutDev
get "/dummy_api" $ do
text $ "dummy response"
I wrote another server which calls this server and returns the response.
{-# LANGUAGE OverloadedStrings #-}
import Web.Scotty
import Network.Wai.Middleware.RequestLogger
import Control.Monad
import Control.Monad.Trans
import qualified Data.Text.Internal.Lazy as LT
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import qualified Data.ByteString.Lazy as LB
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network
main = do
let man = newManager defaultManagerSettings
scotty 3000 $ do
middleware logStdoutDev
get "/filters" $ do
response <- liftIO $! (testGet man)
json $ decodeUtf8 (LB.fromChunks response)
testGet :: IO Manager -> IO [B.ByteString]
testGet manager = do
request <- parseUrl "http://localhost:4001/dummy_api"
man <- manager
let req = request { method = "GET", responseTimeout = Nothing, redirectCount = 0}
a <- withResponse req man $ brConsume . responseBody
return $! a
With both these servers running, I performed wrk benchmarking and got extremely high throughput.
wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
30 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 30.86ms 78.40ms 1.14s 95.63%
Req/Sec 174.05 62.29 1.18k 76.20%
287047 requests in 1.00m, 91.61MB read
Socket errors: connect 0, read 0, write 0, timeout 118
Non-2xx or 3xx responses: 284752
Requests/sec: 4776.57
Transfer/sec: 1.52MB
While this was significantly higher than other web servers like Phoenix, I realized this meant nothing as majority of responses were 500 errors occuring due to file descriptor exhaustion.
I check the limits which were pretty low.
ulimit -n
256
I increased these limits to
ulimit -n 10240
I ran wrk again and this time clearly enough throughput had been reduced drastically.
wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
30 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 105.69ms 161.72ms 1.24s 96.27%
Req/Sec 19.88 16.62 120.00 58.12%
8207 requests in 1.00m, 1.42MB read
Socket errors: connect 0, read 0, write 0, timeout 1961
Non-2xx or 3xx responses: 1521
Requests/sec: 136.60
Transfer/sec: 24.24KB
Although the amount of 500 errors had reduced, they were not eliminated. I benchmarked Gin and Phoenix and they were way better than Scotty
while not giving any 500 responses.
What piece of puzzle I am missing? I suspect there is an issue I'm failing to debug.
I understand that http-conduit has a lot to do with these errors and http-client
library uses it under the hood and this has nothing to do with Scotty
.
Manager
instead of creating it each time? E.g. passManager
instead ofIO Manager
totestGet
. Alsohttp-client
relies onGC
to reuse connections, that may (or may not) explain file descriptor exhaustion. – Heerlenstrace
? – Denewrk
way too much? Maybe that's suffocating the servers? – CollectivizetestGet
fromIO Manager
toManager
solved the entire issue. – Adulterate