Unexpectedly low throughput for Network I/O using Scotty
Asked Answered
A

1

15

I tried to benchmark Scotty to test the Network I/O efficiency and overall throughput.

For this I set up two local servers written in Haskell. One which doesn't do anything and just acts as an API.

Code for the same is

{-# LANGUAGE OverloadedStrings #-}


import Web.Scotty

import Network.Wai.Middleware.RequestLogger 

import Control.Monad
import Data.Text
import Control.Monad.Trans
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network
main = do 
  scotty 4001 $ do
    middleware logStdoutDev
    get "/dummy_api" $ do
        text $ "dummy response"

I wrote another server which calls this server and returns the response.

{-# LANGUAGE OverloadedStrings #-}


import Web.Scotty

import Network.Wai.Middleware.RequestLogger 

import Control.Monad
import Control.Monad.Trans
import qualified Data.Text.Internal.Lazy as LT
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import qualified Data.ByteString.Lazy as LB
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network


main = do 
  let man = newManager defaultManagerSettings 
  scotty 3000 $ do
    middleware logStdoutDev

    get "/filters" $ do
        response <- liftIO $! (testGet man)
        json $ decodeUtf8 (LB.fromChunks response)

testGet :: IO Manager -> IO [B.ByteString]
testGet manager = do
    request <- parseUrl "http://localhost:4001/dummy_api"
    man <- manager
    let req = request { method = "GET", responseTimeout = Nothing, redirectCount = 0}
    a <- withResponse req man $ brConsume . responseBody
    return $! a

With both these servers running, I performed wrk benchmarking and got extremely high throughput.

wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
  30 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    30.86ms   78.40ms   1.14s    95.63%
    Req/Sec   174.05     62.29     1.18k    76.20%
  287047 requests in 1.00m, 91.61MB read
  Socket errors: connect 0, read 0, write 0, timeout 118
  Non-2xx or 3xx responses: 284752
Requests/sec:   4776.57
Transfer/sec:      1.52MB

While this was significantly higher than other web servers like Phoenix, I realized this meant nothing as majority of responses were 500 errors occuring due to file descriptor exhaustion.

I check the limits which were pretty low.

ulimit -n
256

I increased these limits to

ulimit -n 10240

I ran wrk again and this time clearly enough throughput had been reduced drastically.

wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
  30 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   105.69ms  161.72ms   1.24s    96.27%
    Req/Sec    19.88     16.62   120.00     58.12%
  8207 requests in 1.00m, 1.42MB read
  Socket errors: connect 0, read 0, write 0, timeout 1961
  Non-2xx or 3xx responses: 1521
Requests/sec:    136.60
Transfer/sec:     24.24KB

Although the amount of 500 errors had reduced, they were not eliminated. I benchmarked Gin and Phoenix and they were way better than Scotty while not giving any 500 responses.

What piece of puzzle I am missing? I suspect there is an issue I'm failing to debug.

I understand that http-conduit has a lot to do with these errors and http-client library uses it under the hood and this has nothing to do with Scotty.

Adulterate answered 3/6, 2015 at 4:42 Comment(9)
What if you reuse Manager instead of creating it each time? E.g. pass Manager instead of IO Manager to testGet. Also http-client relies on GC to reuse connections, that may (or may not) explain file descriptor exhaustion.Heerlen
I removed the IO wrapper around the manager and tested it again. Doesn't make any difference. There is an underlying issue which I'm not being able to debug.Adulterate
Maybe try running your servers from strace?Dene
Isn't 30 threads in wrk way too much? Maybe that's suffocating the servers?Collectivize
@Collectivize I have found out the issue. Its the same as what @Heerlen Jun told. I'll soon update this post. Changing the signature of the first variable in testGet from IO Manager to Manager solved the entire issue.Adulterate
Could you add your fix as an answer?Scantling
@Adulterate I just hit this unanswered question. Could you please add your fix as an answer like suggested ?Ault
@Ault I have added an answer. If possible, please provide a reason as to why this happened.Adulterate
@Adulterate Perhaps this was a memory leak, but I can't tell you the exact reason. But thanks for providing the answer, you should be able to accept itAult
A
1

@Yuras's analogy was correct. On running the server again, all the issues related to non 2xx status code were gone.

The first line in the main block was the culprit. I changed the line from

main = do 
  let man = newManager defaultManagerSettings

to

main = do 
  man <- newManager defaultManagerSettings

and voila, there weren't any issues. Also the high memory usage of the program stabilized to 21MB from 1GB earlier.

I don't know the reason though. It would be nice to have an explanation for this.

Adulterate answered 22/10, 2015 at 20:11 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.