Zmq Context - Should I create another context in a new thread?

Question

I've got several server apps that use a shared ZMQ class I created. Occasionally when these servers process a request, they need to send a message to another ZMQ server. I'm fairly new to ZMQ so I wanted to make sure I understand this correctly.

The class that handles the server listener creates a zmq::context_t and zmq::socket_t, and binds the socket. This runs on a separate thread infinitely. When this server needs to send a message in another function ( completely torn-off from this ZMQ class ), would I need to generate a new context here and send the message, or should I somehow pass down the same context to this class ( on a different thread ), bind a new socket then go from there?

If the former, does it matter what number I use to initialize the new context, or is context( 1 ) fine? There's a part in the guide that says creating a second context is like having multiple instances of ZMQ, which I don't think really matters since its only being used to send a file then closing. But I'm probably wrong?

Graupel

I cannot tell what is the better practice, but the docs say: »A ØMQ context is thread safe and may be shared among as many application threads as necessary, without any additional locking required on the part of the caller.« – Twinscrew

The context ought never be a "consumable/disposable", it has a lot of overhead associated with its instantiation. The number of IOthreads is another story -- more related with achieving a minimum latency, blocking-state robustness and data-pumping performance ( or performance-class based grouping ). See ZMQ_AFFINITY used for mapping of individual socket(s) onto respective IOthread-#(s). High-performance + low-latency code simply has to tweak these internalities to the max. inproc:// transport-class is IO-less & may use an instance of context( 0 ) it is a memory map. – Hesitate

@HenriMenke you might be interested in ZeroMQ, being a principally locking-free mental concept as the underlying services handle all that automatically. ZeroMQ has set a remarkable yardstick for distributed computing by a few of its maxims -- ZeroCopy, ZeroSharing, ZeroLocking, (almost) ZeroLatency :o) et al -- Pieter Hintjens' book "Code Connected, Volume 1." seems to be a bible for any advanced signalling / messaging in distributed systems. Worth a time & effort to read it ( pdf available too ). – Hesitate

I didn't realize the context # was the IOThreads at first, thought it had something to do with the context identifier since there is a set IOThreads function as well. But I'm working on modifying our apps to all use pointers to the main context now. Thanks for the input. – Graupel

Answer 1

In short: a single instance of context, with a single I/O thread, is probably what you need. In more detail:

Per the docs the context is thread safe. Note that the original 0MQ sockets generally are not ( with having some newer, drafted, ones designed so as they become such ).

From the FAQ:

What is the optimal number of I/O threads for best performance?

The basic heuristic is to allocate 1 I/O thread in the context for every gigabit per second of data that will be sent and received ( aggregated ). Further, the number of I/O threads should not exceed ( number_of_cpu_cores - 1 ).

Another thing to note is inproc:// transport-class sockets must be created in the same context.

Unless you're transferring multiple gigabits/sec, I recommend a single context for your entire process with a single I/O thread.

Also as a guideline, the 0MQ context is intended to be "long-lived". It would be unusual to create/destroy multiple contexts in a single run of an application. This is generally true of 0MQ sockets, too. If you are creating/destroying many sockets you may have taken the wrong approach.

Extravasate

There new thread safe sockets. But there in draft stage for now. – Incomprehension

what about sockets? is it better to save them or to make a new one often? I have zmq sending messages within a function that gets called every some ms - e.g. 50 or 100 – Cyndi