call C++ using Eigen Library function in python

Question

I'm doing some calculations in C++ with help of Eigen Library, the function is like this:

MatrixXd Cov(MatrixXd Data)
{

  VectorXd meanVector;
  ...
  return Covariance;
}

..in the wrap python function:

static PyObject *Wrap_Cov(PyObject *self,PyObject *args)
{   
      Pyobject *Objectdata;

      if(!PyArg_ParseTuple(args,"O", &ObjectData))
        return NULL;

      Cov(ObjectData);

      return Py_BuildValue("O",&covariance_answer);

  }

Obviously, Python doesn't know the ''object'' I defined, it can't translate ''MatrixXd'' to ''Object'', I think it's some kinds of ''array'', not ''object''

How can I do this without using boost?

Rangy

You could use SWIG to generate a C++ wrapper so that Python knows your class' interface. Unfortunately I don't know the specifics of this process, as I have not done it myself (only used a library where it had been done), which is why this is a comment not an answer. – Wolverhampton

Can you confirm that you want to create a C++ extension for Python? In other words you want to make your C++ Cov() function available to your Python script? If so, you basically want to use the PyArg_* function to extract data from the Python function arguments into a C++ variable (not a PyObject as you did above), then use PyBuild* function to pass the result back to Python. If you provide python code that shows an example use case, then we can post an answer. – Iloilo

You mention you can't use boost. But SWIG is a tool that I have used a lot as it generates C++ code that does all the PyArg and PyBuild etc calls for you, with exception propagation etc. All you have to do to use SWIG is install it on your computer, create a .i file that includes the .h of your MatrixXd class and Cov function, run it, build the library that SWIG generated, and start Python: you can then call your Cov from Python you'll have no C API to use at all! – Iloilo

@Schollii: wow thanks! That is a good piece of advice. I will try to keep that in mind for my personal pojects . – Infinitive

Do you want a Cython or SWIG solution or do you want this wrapper written in pure C? – Remscheid

Answer 1

If interfacing numerical modules written in different languages, it's a good practice to keep the data exchanged as flat as possible.

The flattest representation of a libeigen real Matrix is a c array of real type (float or double)

Here's a C++¹ sample

---

#include <stdexcept>
#include <iostream>
#include <string>
#include <sstream>
#include <python2.7/Python.h>
#include <eigen3/Eigen/Dense>


using std::size_t;
typedef double real_t;

typedef Eigen::Matrix<real_t, Eigen::Dynamic, Eigen::Dynamic> 
        Matrix;

static PyObject* p_eigen_python_error(NULL);

static PyObject *
randomDxDMatrix(PyObject *self, PyObject *args) {
    PyObject* p(NULL);
    PyObject* item(NULL);    

    try{
        size_t d(0);

        PyArg_ParseTuple(args, "L", &d);
        Matrix M = Matrix::Random(d,d);

        size_t length = d * d;

        p = PyList_New(length);

        if (p == NULL) {
            std::stringstream msg;
            msg << "Could not allocate a Pylist of "
                << d << "x" << d << " = " << d*d 
                << " size for the return Object";
            throw std::runtime_error(msg.str().c_str());
        } else {
            for (size_t i = 0; i < length; ++i) {
                item = PyFloat_FromDouble(M.data()[i]);
                PyList_SET_ITEM(p, i, item);
            }   
        }

    } catch (const std::exception& e) {
        delete p; p = NULL;
        delete item; item = NULL;

        std::string msg = ("randomDxDMatrix failed: ");
        msg += e.what();
        PyErr_SetString(p_eigen_python_error, msg.c_str());
    }

    return p;
}

static PyMethodDef EigenMethods[] = {
    {"randomDxDMatrix",  randomDxDMatrix, METH_VARARGS, 
    "Gets a random DxD matrix column-major as a list of (python) floats"},
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

PyMODINIT_FUNC
initeigen_python(void) {

    PyObject* p;

    p = Py_InitModule("eigen_python", EigenMethods);
    if (p == NULL)
        return;

    p_eigen_python_error = PyErr_NewException(
                                const_cast<char*>("eigen_python.error"), 
                                NULL, NULL
                            );
    Py_INCREF(p_eigen_python_error);
    PyModule_AddObject(p, "error", p_eigen_python_error);
}

---

Here's setup_eigen_python.py

---

from distutils.core import setup, Extension

cpp_args = ['-Wall', '-pedantic']
cxx_args = ['-std=c++11'].extend(cpp_args)

module_eigen_python = Extension('eigen_python',
                          define_macros = [('MAJOR_VERSION', '0'),
                                            ('MINOR_VERSION', '1')],
                          include_dirs = ['/usr/local/include'],
                          sources = ['eigen_python.cpp'],
                          extra_compile_args = cpp_args
#                          sources = ['eigen_python.cxx'],
#                          extra_compile_args = cxx_args
                      )

setup (name = 'eigen_python',
       version = '0.1',
       description = 'This is just a demo',
       author = 'Solkar',
       url = 'http://stackoverflow.com/questions' 
         + '/15573557/call-c-using-eigen-library-function-in-python',
       long_description = 'just a toy',
       ext_modules = [module_eigen_python])

---

to be used like

python2.7 setup_eigen_python.py install --user

and here's a little test driver

---

import eigen_python as ep
import numpy as np

DIM = 4

M = np.array(ep.randomDxDMatrix(DIM), order="F")
M.shape= DIM,DIM

print(M) 

---

¹Especially, but by far not restricted to, because having to get along without boost, would have preferred using features of the C++ 2011 standard like auto and std::unique_ptr but I did not know if the QO has sufficient support for this.

Mottled