How to send byte message with ZeroMQ PUB / SUB setting?

Question

So I'm new to ZeroMQ and I am trying to send byte message with ZeroMQ, using a PUB / SUB setting.

Choice of programming language is not important for this question since I am using ZeroMQ for communication between multiple languages.

Here is my server code in python:

import zmq
import time

port = "5556"
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:%s" % port)
while True:
    socket.send(b'\x84\xa5Title\xa2hi\xa1y\xcb\x00\x00\x00\x00\x00\x00\x00\x00\xa1x\xcb@\x1c\x00\x00\x00\x00\x00\x00\xa4Data\x08')
    time.sleep(1)

and here is my client code in python:

import zmq

context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:5556")

total_value = 0
for update_nbr in range (5):
    string = socket.recv()
    print (string)

My client simply blocks at string = socket.recv().

I have done some study, so apparently, if I were to send string using PUB / SUB setting, I need to set some "topic filter" in order to make it work. But I am not sure how to do it if I were to send some byte message.

Save

Well I solved the problem myself. Pub-Sub-Message-Envelopes need to be used. zguide.zeromq.org/page%3aall#Pub-Sub-Message-Envelopes – Save

Well, while a use of (an artificial) message data split into a dual-frame'd "envelope" is a possible side-step from the main problem & remains possible, it has several non-zero add-on costs to do so - a) the messages cease to be native PUB/SUB messages (some programming language bindings happened to have done this side-step in silence, so receiving SUB-s need not be aware of to process an arriving message (promoted to be plain PUB/SUB one) as a multi-frame composition & cross-platform solutions turn wreck havoc)+ b) refactoring. Byte-correct subscribes are a clean solution at zero-cost added. – Krever

@user3666197. Thank you. What is "Byte-correct subscribes are a clean solution at zero-cost added." please? – Himself

Answer 1

ZeroMQ defines protocols, that guarantee cross-platform compatibility of both the behaviours and the message-content .

The root-cause:
_{to start receiving messages
one must change the initial "topic-filter" state for the SUB-socket
( which initially is "receive nothing" Zero-subscription )}

ZeroMQ is a lovely set of tools, created around smart principles.

One of these says, do nothing on SUB-side, until .setsockopt( zmq.SUBSCRIBE, ... ) explicitly says, what to subscribe to, to start checking the incoming messages ( older zmq-fans remember the initial design, where PUB-side always distributes all messages down the road, towards each connected SUB-"radio-broadcast receivers", where upon each message receival the SUB-side performs "topic-filtering" on it's own. Newer versions of zmq reverse the architecture and perform PUB-side filtering ).

Anyway, the inital-state of the "topic-filter" makes sense. Who knows what ought be received a-priori? Nobody. So receive nothing.

Given you need or wish to start the job, an easy move to subscribe to anything ... let's any message get through.

Yes, that simple .setsockopt( zmq.SUBSCRIBE, "" )

If one needs some key-based processing and the messages are of a reasonable size ( no giga-BLOBs ) one may just simply prefix some key ( or a byte-field if more hacky ) in front of the message-string ( or a payload byte-field ).

Sure, one may save some fraction of the transport-layer overhead in case the zmq-filtering is performed on the PUB-side one ( not valid for the older API versions ), otherwise there is typically not big deal to subscribe to receive "anything" and check the messages for some pre-assembled context-key ( a prefix substring, a byte-field etc ) before the rest of the message payload is being processed.

---

The Best Next Step:

If your code strives to go into production state, not to remain as just an academia example, there will have to be much more work to be done, to provide surviveability measures for the hostile real-world production environments.

An absolutely great perspective for doing this and a good read for realistic designs with ZeroMQ is Pieter HINTJEN's book "Code Connected, Vol.1" ( may check my posts on ZeroMQ to find the book's direct pdf-link ).

Plus another good read comes from Martin SUSTRIK, the co-father of ZeroMQ, on low-level truths about the ZeroMQ implementation details & scale-ability

Krever

Yes, that simple .setsockopt( zmq.SUBSCRIBE, "" ) Actually, for subscribing to binary messages, the second parameter must be b"" not "". I just tested it. – Elastin

@RestInPeace well, that is not a ZeroMQ property, that is a side-effect of how Py3 started to handle the same thing in a different manner, that the Py2.x did for ages... The most sure & stable thing is the past ... The future? Well, who knows ... :o) – Krever

Oh I see. I have only used python 3.6-3.8 so I can't say about python 2, but I'm just letting you know that "" actually crashed my program and only b'' and b"" work for my current python version which is 3.8. For setsockopt_string though, "" works well. This actually makes sense since setsockopt receives bytes and setsockopt_string receives string. – Elastin