how is cherrypy working? it handls requests well compared with tornado when concurrence is low

Question

I carried out a test on cherrypy (using web.py as a framework) and tornado retrieving webpages from the internet.

I have three test cases using siege to send requests to server (-c means number of users; -t is testing time). Code is below the test results.

1. web.py (cherrpy)

  siege ip -c20 -t100s             server can handle 2747requests  
  siege ip -c200 -t30s             server can handle 1361requests
  siege ip -c500 -t30s             server can handle 170requests

2. tornado synchronous

  siege ip -c20 -t100s             server can handle 600requests  
  siege ip -c200 -t30s             server can handle 200requests
  siege ip -c500 -t30s             server can handle 116requests

3. tornado asynchronous

  siege ip -c20 -t100s             server can handle 3022requests  
  siege ip -c200 -t30s             server can handle 2259requests
  siege ip -c500 -t30s             server can handle 471requests

performance analysis:

tornado synchronous < web.py (cherrypy) < tornado asynchronous

Question 1:

I know, using an asynchronous architecture can improve the performance of a web server dramatically.

I'm curious about the difference between tornado asynchronous architecture and web.py (cherry).

I think tornado synchronous mode handles requests one by one, but how is cherrypy working, using multiple threads? But I didn't see a large increase of memory. Cherrypy might handle multiple requests concurrently. How does it solve the blocking of a program?

Question 2:

Can I improve the performance of tornado synchronous mode without using asynchronous techniques? I think tornado can do better.

Web.py code:

import web
import tornado.httpclient
urls = (
    '/(.*)', 'hello'
)
app = web.application(urls, globals())

class hello:
    def GET(self, name):
        client = tornado.httpclient.HTTPClient()
        response=client.fetch("http://www.baidu.com/")
        return response.body

if __name__ == "__main__":
    app.run()

Tornado synchronous:

import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        client = tornado.httpclient.HTTPClient()
        response = client.fetch("http://www.baidu.com/" )
        self.write(response.body)


if __name__=='__main__':
    tornado.options.parse_command_line()
    app=tornado.web.Application(handlers=[(r'/',IndexHandler)])
    http_server=tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

Tornado asynchronous:

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient
from tornado.options import define, options
define("port", default=8001, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        client = tornado.httpclient.AsyncHTTPClient()
        response = client.fetch("http://www.baidu.com/" ,callback=self.on_response)

    def on_response(self,response):
        self.write(response.body)
        self.finish()

if __name__=='__main__':
    tornado.options.parse_command_line()
    app=tornado.web.Application(handlers=[(r'/',IndexHandler)])
    http_server=tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

Nuisance

I very much suspect that your performance may be limited by fetch('http://www.baidu.com/') much more than by the web frameworks you use. Try comparing things when you serve static content, or, at least, the same locally-generated content. – Peterus

it doesn't matter. I'm comparing the performance of different web server."fetch('baidu.com/')" is just a blocking code. – Nuisance

Where does cherrypy come into this? – Vogt

web.py uses cherrypy as built-in httpserver – Nuisance

Answer 1

To answer question 1...

Tornado is single threaded. If you block the main thread, as you do in your synchronous example, then that single thread cannot do anything until the blocking call returns. This limits the synchronous example to one request at a time.

I am not particularly familiar with web.py, but looking at the source for its HTTP server it appears to be using a threading mixin, which suggests that it is not limited to handling one request at a time. When the first request comes in, it is handled by a single thread. That thread will block until the HTTP client call returns, but other threads are free to handle further incoming requests. This allows for more requests to be processed at once.

I suspect if you emulated this with Tornado, eg, by handing off HTTP client requests to a thread pool, then you'd see similar throughput.

Mononuclear

When i was testing web.py,I have been observing the changes in memory which cherrpy took up。I know,normally, a thread takes up 8m in linux.Meanwhile，at the beginning ,cherrypy takes up 12m and when it was handling requests,it only took 15m.I don't think it uses multi thread.that is why i am curious about cherrypy. – Nuisance

Answer 2

Most of the handler time in your test code is by far spent in client.fetch(...) - effectively waiting for connection and for incoming data on a socket - while not blocking potential other Python threads.

So your "performance measure" is mostly determined by the max number of effective handler threads of the framework in question and by the max number of parallel connections which the "baidu" server allows from your IP.

wep.py's copy of CherryPyWSGIServer web.wsgiserver.CherryPyWSGIServer which is used by the default web.httpserver.runsimple() indeed uses threads - 10 by default.
Threads do not increase memory usage a lot here. Most memory is consumed by the libraries and Python interpreter itself here. And CherryPyWSGIServer's (10) handling worker threads are all started right at the beginning. The alternative web.httpserver.runbasic() also uses threads - via Python's builtin HTTPServer and SocketServer.ThreadingMixIn. This one starts a new thread for each request. Probably "unlimited" number of threads - but there is overhead for thread startup for each request.

tornado asynchronous mode may also use more/unlimited number of threads (?), which may explain the difference to web.py here.

Your test doesn't say much about the execution speed of the servers & handler frameworks themselves. You may simply increase the max number of threads in web.py's CherryPyWSGIServer. Parallel execution of your client.fetch(...)'s is somehow needed to get more "performance" here.

To test the mere server / framework speed (the overhead cost) simply return a string or a database query or a typical complete web page rendered from local contents.

A multithreaded CPython based web&app server in one process finally cannot use much more than one CPU core (of maybe 8 CPU cores typically today on server hardware) - because of the GIL in CPython, which is only released for some I/O. So if CPU-load becomes a factor (and not the network or database speed), Jython or a multi-processes approach could be considered.

Knack