mpi4py: Replace built-in serialization

Question

I'd like to replace MPI4PY's built-in Pickle-serialization with dill. According to the doc the class _p_Pickle should have 2 attributes called dumps and loads. However, python says there are no such attributes when i try the following

from mpi4py Import MPI
MPI._p_Pickle.dumps 

-> AttributeError: type object 'mpi4py.MPI._p_Pickle' has no attribute 'dumps'

Where have dumps and loads gone?

Smithson

maybe I'm using an older Version? how 2 check mpi4py version? When I try "MPI.VERSION" it says 2... MPI.Get_Version gives (2,1) which corresponds to (Version,Subversion). Im quite confused since the latest Version is 1.3. What the... – Smithson

Maybe MPI.Version tells you the version of the underlining MPI implementation. Are you working with MPICH (newest version :3.0) ? or openmpi (1.7) ? or ... What is the output of mpicc -show ? – Groh

Answer 1

In v2.0 you can change it via

MPI.pickle.dumps = dill.dumps
MPI.pickle.loads = dill.loads

It seems that the documentation is still from 2012.

Update For v3.0 see here, i.e.:

MPI.pickle.__init__(dill.dumps, dill.loads)

Liman

Answer 2

You are probably using an older version. Use 1.3.1 not 1.2.x. Check the version number with mpi4py.__version__. If you are using 1.3.1 or newer, you can overload dumps and loads with serialization from dill, or cloudpickle, or a some other custom serializer.

>>> import mpi4py
>>> mpi4py.__version__
'1.3.1'

Unknit