Due to having trouble with the reliability and scheduling in celery we decided to evaluate alternatives. I have been struggling setup a benchmark between the two message queue solutions with regards to base performance.
My current approach is to place 1000 tasks (of getting nvie.com and counting the words on the site) on the two different queues and measuring how fast 4 celery (20 sec) vs 4 rq workers (70 sec). My code is https://github.com/swartchris8/celery-vs-rq-benchmark I am running celery through the command line and rq through supervisor on a Mac, Ubuntu running instructions for rq are clear from the vagrant file.
Celery is performing so much better and I am not sure if my test setup of measuring how fast the queues are clearing is flawed for measuring task throughput. I am also using default RQ workers which I suspect could be much slower.
Is my approach the correct way in benchmarking the two message queue system with regards to throughput? What kind of approaches have you taken? Is celery so much faster than RQ?