Erlang (or elixir) performance (requests per second) is slow vs jruby?

Question

Being a rubyist, I decided to take on erlang for high performance, reliable backend. The setup is quite simple: get a post request, write stuff to redis, return statistics. All json. this is also why I care so much about requests per second.

Tools of choice: webmachine, jiffy for json encoding/decoding, poolboy for connection pool, and eredis for redis communication.

Machine used: macbook pro, i5 2.4Ghz, 8GB memory.

My erlang got about 5000 requests per second, and the jruby/torqbox got about 12,0000. (look here for a complete ruby performance test setup)

I realize I could use ets in erlang to save time, and leave the redis for 'background processing' to be written after the response, but this will have little impact. even a simple test of 'hello world' erlang legs behind.

Any suggestions? Am I doing it wrong?

Postdate

What are you using with jruby? Note that webmachine is not a web server. It is closer to a web library/framework. If you are using just a web server with jruby (or a straight-forward rack application) than you should consider using Cowboy directly for a more apple to apples comparison. For reference, I get about 11000 req/s with Cowboy on an old Macbook Pro. – Maurilia

And here is another benchmark you could use for reference (maybe try running them with jruby?): github.com/mroth/phoenix-showdown – Maurilia

As comparison you could take a look on the results and on the source code of these benchmarks: techempower.com/benchmarks – Osteal

Why did you tag this with Elixir? Are you using Elixir code or Erlang code? It may make a difference as far as the answers people give. – Inequity

Hi all, and thanks so much for being so responsive. @JoséValim: I realize that webmachine runs on mochiweb, but that was compared to a ruby Grape, on the new torqbox. so it is apples to apples. why tagged elixir? - this is exactly because I tried many solutions, and the phoenix/elixir was one of them. it performed slightly better than the webmachine solution as described. Thanks for the useful links. will look into. I was under the impression that erlang for it's efficiency, would beat the JVM. was surprised to find it didn't – Postdate

Answer 1

• Webmachine - I don't know. It is pretty heavy weight for mine taste and I don't use it.
• jiffy - Good choice. Pretty fast and I use it a lot.
• poolboy - I don't heard about it and I'm around Erlang for several years. I definitely would use cowboy for anything which I would expect high-performance or yaws for some more rock solid but still performing good. For your benchmark is cowboy right choice, it is top performance-wise.
• eredis - I don't know how mature and how it is efficient. ets is more appropriate for benchmark. For your macbook test it doesn't matter but for for big server (tens of CPUs) I would check if ti is not bottleneck and partition table.
• most importantly check your VM parameters. At least you should have +K true +A 100 for this kind of load.

Your result for Erlang seem simply too low compared to mine experience. You should get almost ten times bigger. There also can be problem with your payload generating tool.

And mostly important, this can be considered not only micro benchmark for Erlang world but may be nano or atto benchmark. The real strength will reveal when you will try something way harder and more complicated. Something where concurrent request should affect each other in very complicated way and you have to deal with eventual consistency and implement it much more scalable and need use asynchronous inter process communication.

Haya

Nice write up! Just a note: poolboy is for connection pooling not for a web server. It is used by Riak and a couple other places so I think it is really standard. – Maurilia

About the pooling - has anyone got other valid methods to approach the subject of database connections (redis in my case)? please do elaborate. While at it please recommend a solid redis or similar solution for fast persistence. I appreciate @hynek-pichi-vychodil last paragraph, and want to get into Erlang/Elixir for that reason. but for my current task, which is dead simple, performance is top priority, and I probably will have hard time justifying a new tool, if performance is not way better (we mostly work in ruby/jruby). – Postdate

Answer 2

My 2 cents. You have the wrong end of the stick. You are testing a problem that the JVM is optimized for on a kind of machine that the JVM is optimized to work on.

You really aren't going to see the advantages of Erlang/OTP on the number of cores available on a mac book pro. This is just my rough guess, but I would be surprised to see Erlang beat the JVM on anything less than a 8 core server. There has been a tremendous number of man/hours put into making the JVM as fast as possible on current hardware.

Writing Thread safe I/O code is fairly straightforward, the real problems arise when you're dealing with memory access among tens to hundreds of threads.

Writing in Erlang/Elixir is aiming at the current high end server of 16 or 32 cores with the potential to scale much higher in the near future.

Luteous

Just something you might want to see. littlelines.com/blog/2014/07/08/… PS: I have nothing to do with the benchmarks, just found them interesting. – Plasticizer

Answer 3

FYI: "+A 100" wouldn't help with networking, it's only for file IO. If you really want fast web server take a look at github.com/knutin/elli it will give you 80 kprs on the hardware where cowboy will give you 30 krps. On the other hand, elli is something many people are blaming for not following OTP principles.

jiffy is a nice choice if you can place some load balancer to sanitize requests because jiffy will introduce segfaults to your code - take a look at issue list.

anyway erlang is not something you may want to choose if all you need is really fast GET -> decode json -> store -> REPLY workload.

Walleyed