How do I read and write with msgpack?

Question

How do I serialize / deserialize a dictionary data with msgpack?

Afore

Answer 1

The Python docs seem not to be so good, so here is my try.

Installation

pip install msgpack

Read and Write msgpack

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import msgpack

# Define data
data = {
    "a list": [1, 42, 3.141, 1337, "help"],
    "a string": "bla",
    "another dict": {"foo": "bar", "key": "value", "the answer": 42},
}

# Write msgpack file
with open("data.msgpack", "wb") as outfile:
    packed = msgpack.packb(data)
    outfile.write(packed)

# Read msgpack file
with open("data.msgpack", "rb") as data_file:
    byte_data = data_file.read()

data_loaded = msgpack.unpackb(byte_data)
print(data == data_loaded)

Alternatives

• CSV: Super simple format (read & write)
• JSON: Nice for writing human-readable data; VERY commonly used (read & write)
• YAML: YAML is a superset of JSON, but easier to read (read & write, comparison of JSON and YAML)
• pickle: A Python serialization format (read & write)
• MessagePack (Python package): More compact representation (read & write)
• HDF5 (Python package): Nice for matrices (read & write)
• XML: exists too *sigh* (read & write)

For your application, the following might be important:

• Support by other programming languages
• Reading / writing performance
• Compactness (file size)

See also: Comparison of data serialization formats

In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python

Afore

We now only use pip install msgpack. msgpack-python is outdated, as per pypi. – Martita

What is the recommended extension for msgpack file? – Asterism

Well, I would recommend msgpack. But I'm also not aware of any other Python packages that support msgpack. – Afore

msgpack._packer.Packer._pack TypeError: can't serialize 1.0 – Asterism

For me when using more recent versions of python a rb flag has to be added to the file open function (I have read, not sure about writing). – Nevlin

Getting error TypeError: write() argument must be str, not bytes. I am on Py3.7 – Damp

@Damp Thank you for letting me know. I'm not sure why, but the code didn't work anymore. I've updated the example. – Afore

what if i dont have the data and i need to paste the jiberish from redis to the load, who to do this? – Pumping

are the msgpack in your answer and the MessagePack in the "alternatives" the same? – Polyvalent