zeromq: unicode not allowed, use send_unicode

Question

I am getting error on zeromq python while sending strings through ROUTER SOCKET. String type messages are receveid successfully but some times, a unicode message throws exception "Type Error: unicode not allowed. use send_unicode". Although I have been trying to use msg.encode('utf-8'). BUt I cant figure out a way to get over with it. I am on python 2.7.3. I am not using pyzmq (import zmq only). Looking forward to your suggesitons :) Thanks

if backend in sockets:

        request=backend.recv_multipart()
        #print ("Backend Thread is ready")
        worker_id,client_id = request[:2]

        if client_id != b"READY" and len(request) > 3:
            #print (len(request))
            empty2,reply  = request[2:]
            router_socket.send_multipart([client_id, reply.encode('utf-8')])

Filipino

I have also used router_socket.send (client_id,zmq.SNDMORE) router_socket.send_unicode(reply.encode('utf-8')) but still same error goes – Filipino

Answer 1

I got the same error. My erroneous code was:

socket.send("Server message to client3")

You must convert the message to bytes to solve it. To do so, just add b like this:

socket.send(b"Server message to client3")

Is it better to convert strings to byte, then bytes to strings when data sent through network, and why?

Sauers

Answer 2

The problem was resolved only thing was that I needed to convert the unicode strings back to ascii by using string.encode('ascii')

Filipino

That's only going to work for the subset of Unicode that is also ASCII. – Paisa

what alternative do you suggest me to use? @UlrichEckhardt – Filipino

It's a bit difficult to tell, since I can't locate e.g. that send_unicode() function that you mentioned, so I'm not sure we're talking about the same API. Now, the point is that either you do the encoding and then let ZMQ transmit bytes or you give Python unicode strings to ZMQ and let it handle the encoding internally. If you want to handle it yourself, using ASCII as encoding is not able to represent the full Unicode range, which is often not desired. BTW: You should create a minimal example, because that helps you understand the issues. E.g. which parameter causes the type error? – Paisa

@UlrichEckhardt The problem was resolved since zeromq has builtin unicode characteristics as mentioned here: socket.send_string(self, unicode s, flags=0, encoding='utf-8') takes a unicode string s, and sends the bytes after encoding without an extra copy, via: socket.send(s.encode(encoding), flags, copy=False) – Filipino

Answer 3

So, because PyZMQ is actually a good library they have some docs.
https://pyzmq.readthedocs.io/en/latest/unicode.html

What it says is that the str object changed it's nature over the course of history of Python evolution.
In Python 3 str is a collection of characters and in Python 2 it is a simple wrapper (with some sugar) for char* that we know from C :).
Docs explain why the people behind pyZMQ chose to make the differences explicit - performance is the answer.
To send strings in Python3 you should use the right method, which is send_string, probably the other way around for Python2 (to send unicode you should use send_unicode).
It is however recommended to stick to bytes and explicitly provide correct encoding and decoding where needed.

Also you are using pyzmq... the module name "zmq" comes from pyzmq library/package.
To confront this statement use: pip list | grep zmq (or pip list | select-string zmq for Windows).

Dicotyledon