zmq_ctx_term() blocks while sockets closed

Question

I am working with ZMQ and am running into issues when destroying a context.

I have the code

zmq_ctx_shutdown(context);
zmq_ctx_term(context);

This code always blocks on zmq_ctx_term() and will also block with zmq_ctx_destoy(); however, if that call is removed and only the shutdown call is used, everything seems to work fine, but the application will leak memory from not freeing the zmq context.

In my code, I have four sockets: two ZMQ_PAIR inproc sockets that are used to communicate between the main thread and a thread with a ZMQ_REP socket, along with a ZMQ_PUB that runs on the main thread.

I have set ZMQ_LINGER on all of these sockets to 0, as it seems the only time the zmq_ctx_term() call should block is if messages have not yet been sent since I have called zmq_close() on all sockets, which returns without errors. Additionally, this blocking still occurs if I only call zmq_ctx_new(), create the sockets, then call zmq_ctx_shutdown(); zmq_ctx_term();.

My fear is that something is wrong with using the two inproc sockets to communicate between threads with the same zmq context, although there doesn't seem to be any issues with the communication as messages are being received.

After fearing that I may have some issue with thread safety, I found that the ZMQ context should be thread-safe, but not the sockets. I have checked and in the main thread, I am opening an closing the ZMQ_PUB socket and inproc socket. In another thread, I am opening and closing the ZMQ_REP socket and other end of the inproc socket, so it seems like that shouldn't be an issue.

For clarification, I am writing my code in C using libzmq from the current master branch on GitHub (commit d35473e). I did see a similar on issues when linking ZMQ's shared library, although this is occurring whether I use the static or shared library. I am currently on OS X 10.9.

If anyone has time to look at the code as a whole, the relevant file is listed here.

Any idea of what is going on here?

Bastia

While the documentation says (on zmq_ctx_shutdown) "This function is optional, client code is still required to call the zmq_ctx_term function to free all resources allocated by ZeroMQ.", I'm running into troubles too. I'm currently not doing zmq_ctx_term if I did zmq_ctx_shutdown. – Deep

@Deep I wasn't running into any issues other than a memory leak by not calling zmq_ctx_term and in fact thought I was doing the right thing until I read the documentation for zmq_ctx_shutdown. It does seem like you shouldn't be setting the context to a null pointer in your Shutdown call though since it still needs to be terminated. – Bastia

What does the stacktrace looks like when blocked on zmq_ctx_term() ? – Stalinsk

@Stalinsk The last call I can see regardless of the case are poll() and epoll() calls, which I would think are coming from a blocking socket, although I am having to send a SIGINT to stop execution, so I'm not sure if that is affecting anything. At the moment, zmq debugging symbols are not showing up for me even if I build in debug mode, so I'm trying to figure that out to see what else is going on. – Bastia

Answer 1

Observations:

I had the same problem. Despite the fact that I was closing the socket of ZMQ_REQ type, and zmq_close() returned 0, zmq_ctx_term() would still block. This was happening only after sending a message to a peer that was not running.

zmq_send() would return immediately, but zmq_recvmsg() would time out.

After that I was closing the socket and calling zmq_ctx_term(). Apparently, despite the fact that zmq_close() returned 0, the sent message was still in an outgoing queue, and for that reason zmq_ctx_term() would block.

Solution:

I solved the problem by setting ZMQ_LINGER option to half the value of ZMQ_RCVTIMEO. This behavior is actually described here http://api.zeromq.org/4-0:zmq-ctx-term

Forsworn