What are the alternatives to ZeroMQ for moving protocol buffer payloads around?

Question

At the moment I have a solution that uses ZeroMQ to exchange protocol buffer payloads. The protocol buffer method of serialization is bound to stay as it is, but I can replace ZMQ with a more convenient option. The things I am not happy about in ZMQ are:

It uses JNI on the Java side,and I've been bitten before by JNI, in complex, multi thread scenarios. I try to eliminate it whenever I can.

I don't need queuing, I just need rpc.

My requirements (which are mostly covered by ZeroMQ) are:

• Support for 32/64 bit *nix, Windows, MacOS.

• Support for Java, C++ and C# primarily, and Python, Ruby etc. would be nice.

• Language support must be provided by native implementations in the language, not via wrapping native code.

• High performance.

• Non Viral license, no GPL, AGPL etc.

• I've been thinking about using Thrift as the transport layer over TCP (I guess it supports that) with protocol buffers payloads, if its Java implementation for messaging is not using JNI.

What options can you think of other than ZMQ for this setup?

Dorton

Have you looked closely at Thrift? Last time I looked at it, it didn't do -just- transport: it's a replacement for protocol buffers (plus transport). – Thaxton

It isn't clear what you really need from 0mq. It's pretty generic (looks almost like straight sockets), so it covers a lot of gorund. You looking for queuing? RPC? – Cannot

@GregKopff : I can't replace protocol buffers in this project. It will have to be used for serialization, no matter what the transport method is – Dorton

@ChristopherSmith : it is the cross platform, cross language rpc that I'm looking for. I've edited the question to make it (hopefully) more clear. – Dorton

@sarikan: understood - hence my comment ... last time I looked at Thrift, it was all geared around using the Thrift IDL to define the message payloads, rather than a cross-platform transport mechanism. – Thaxton

@GregKopff : you may want to check it again :) Unless I'm really not getting it, it seems to support client & server construction too. Please see the example on thrift.apache.org – Dorton

May be a very late answer, Have you checked full java implementation of ZeroMQ github.com/zeromq/jeromq – Doralynne

Have you considered ZeroC ICE? It has IDL which is better than CORBA and mappings for all languages maybe except Ruby. All OS you have mentioned are supported. You should be able to use it as RPC middleware with no problem in no time. – Iphigeniah

Answer 1

Have you considered something like Storm or Spread?

Cannot

Thank you, both are quite interesting. These are new options for me to look at. – Dorton

Note that Storm actually uses ZeroMQ as its transport, hence not an option if you want to replace ZeroMQ. – Ill

Looks like storm now used Netty instead of zeromq : github.com/nathanmarz/storm/pull/556 – Isochromatic

@Isochromatic Not "instead of" but rather "as an option to". "Storm now supports pluggable message transport layer: 0MQ or Netty." from the 0.9.0-rc2 changelog posted to the Storm ML – Maximinamaximize

-1 for claiming that Storm is an alternative to ZeroMQ. Storm is built on top of 0MQ, so it is not a substitute. Therefore, this should not be the accepted answer. – Maximinamaximize

@DavidJames as per rjha94's comment, Storm does not require the use of 0MQ and would appear to address the other issues highlighted by the requirements. Also, Storm's specific 0MQ implementation is locked down and very broadly tested, so the problems identified by the original post would appear to have been mitigated (at least in my experience). – Cannot

Answer 2

You should probably have a look at Netty. It's a high performance Java NIO server framework with built-in support for Protocol Buffer which is released under the terms of the Apache License. The framework is well documented and some examples show how to prototype protocols with Protocol Buffers.

Mcatee

Answer 3

The original question was asked about a year after JeroMQ was put onto github. It is the pure-java implementation of ZeroMQ. It has seen constant development throughout the intervening years and seems to be comparable in speed to the C-implementation.

Scarabaeoid