Sending an existing dict through zmq ipc

Question

I'm trying to send an existing dict through zmq ipc socket, I can send a string with this code, but I can't send a dict object

import zmq, datetime

d = {0: ('356612022462768', 'EVENT', 0, '2012-12-26 15:50:16', -20.22216, -70.13723, 6.44, 134.0, 1, 2, '18743230', datetime.datetime(2013, 2, 10, 9, 6, 2, 362734))}

if __name__ == "__main__":
    context = zmq.Context()
    publisher = context.socket(zmq.PUB)
    publisher.connect("ipc://shared")
    while True:
        publisher.send( d )
        time.sleep( 1 )

---

TypeError: {0: ('356612022462768', 'EVENT', 0, '2012-12-26 15:50:16', 
           -20.22216, -70.13723, 6.44, 134.0, 1, 2, '18743230', 
           datetime.datetime(2013, 2, 10, 9, 6, 2, 362734))} 
does not provide a buffer interface.

How can I do that?

Wacker

Answer 1

It is only possible to either send strings or byte arrays through ZeroMq, out-of-the-box. Anything else needs to be serialized before passing it to ZeroMq for transfer on the wire.

You can use whatever you like to serialize it, for example Protocol Buffers, JSON or Message Pack. Note that any recipients needs to be able to deserialize the data using the same protocol.

You can find examples on how to use various serialization techniques (including a numpy array) in conjunction with pyzmq here. They are part of the pyzmq source.

Psaltery

Answer 2

You'd need to serialise the data, probably to JSON depending on the use case. you can't send it as it is, you'll need a string representation

import json
myjson = json.dumps(d)

But the datetime object cannot simply be converted to json so you'll have to deal with that separately, this post will help with that: JSON datetime between Python and JavaScript

Lupine

Answer 3

Here the solution to your problem:

import zmq, datetime
import json

d = {0: ('356612022462768', 'EVENT', 0, '2012-12-26 15:50:16', -20.22216, -70.13723, 6.44, 134.0, 1, 2, '18743230', datetime.datetime(2013, 2, 10, 9, 6, 2, 362734))}

if __name__ == "__main__":
    context = zmq.Context()
    publisher = context.socket(zmq.PUB)
    publisher.connect("ipc://shared")
    while True:
        publisher.send_string(json.dumps(d, default=str))
        time.sleep( 1 )

The default=str is because datatime is not JSON serializable. It is nasty, for a more clean solution, you can implement this solution:

import zmq, datetime
import json


d = {0: ('356612022462768', 'EVENT', 0, '2012-12-26 15:50:16', -20.22216, -70.13723, 6.44, 134.0, 1, 2, '18743230', datetime.datetime(2013, 2, 10, 9, 6, 2, 362734))}

def json_serial(obj):
    """JSON serializer for objects not serializable by default json code"""
    if isinstance(obj, (datetime.datetime, datetime.date)):
        return obj.isoformat()
    raise TypeError ("Type %s not serializable" % type(obj))

if __name__ == "__main__":
    context = zmq.Context()
    publisher = context.socket(zmq.PUB)
    publisher.connect("ipc://shared")
    while True:
        publisher.send_string(json.dumps(d, default=json_serial))
        time.sleep( 1 )

ATTENTION: json.dumps gives you a string, so you should use send_string and not send.

To read the data you should do import json; json.loads(socket.recv()).

Marv