Creating a numpy array of custom-class objects with C API

Question

Using the C API, I would like to create a numpy array containing objects of type Quaternion, which is a class I've written in C++. I already have an array of these (actually a std::vector), and I want to make a copy -- or use the same memory if possible.

Since this isn't a basic type, I need to use Py_Object types, and can't use PyArray_SimpleNew or anything easy like that.

I'm guessing I might want to use PyArray_NewFromDescr or even PyArray_SimpleNewFromDescr, but I am completely and utterly lost as to how I might create the PyArray_Descr object I need to describe my Quaternion class.

Can anyone give me some pointers about how to make that descr object? Or give me a better idea of how to construct my numpy array?

This is basically a more general version of this question, without the distractions.

EDIT:

Using dastrobu's hint, and my SWIG wrapper, I found a way to do it. I know that not everyone is using SWIG, but for those who are, my answer on my other question shows how I worked it out.

Crematorium

Answer 1

Since Quaternion is not directly a numeric type, your array must have numpy.object as dtype. Hence, you can use PyArray_SimpleNew(..., NPY_OBJECT) to create an array and fill in the data. The problem is that your Quaternion class is not a python type. So filling the array with references to objects of type Quaternion will not work. (In this case, what would you expect to happen, if you extract an element from the array filled with quaternions from python?) Instead, you need to wrap the Quaternion class with something like PyQuaternion. The wrapper takes care of reference counts and memory management. It will look something like:

typedef struct {
    PyObject_HEAD
    Quaternion *q;
}PyQuaternion;

static PyTypeObject PyQuaternion_Type = {
    PyObject_HEAD_INIT(NULL)
    0,                                        /*ob_size*/
    "Quaternion",                             /*tp_name*/
    sizeof(PyQuaternion),                     /*tp_basicsize*/
/* ... */
};


static PyObject *
PyQuaternion_new(PyTypeObject *type, PyObject *args, PyObject *kwds){
/* ... */
};

static int 
PyQuaternion_init(PyQuaternion *self, PyObject *args, PyObject *kwds){
/* ... */
};

static void PyQuaternion_dealloc(PyQuaternion *self){
/* ... */
};

Furthermore you can define your own C-API for the PyQuaternionType allowing you to create PyQuaternions from Quaternions

static PyObject *
PyQuaternion_New(Quaternion *q){
    PyQuaternion *self;
    self = (PyQuaternion *)PyQuaternion_Type.tp_new(type, NULL, NULL);
    self->q = q; 
    return (PyObject *)self;
}

Be aware that self->q will be handled by the PyQuaternion_dealloc function, so think about the memory management. The simplest would be to pass ownership to the wrapper and let PyQuaternion_dealloc deallocate self->q.

The PyQuaternion_New function allows you to wrap Quaternion objects and fill them in to any python container, like lists, tuples and of course also numpy arrays with dtype = numpy.object.

Swordcraft

Thanks. That makes a lot of sense. And I'm actually doing all this through SWIG, so I'm sure it's wrapped Quaternion for me as some type of PyObject. Now I just have to figure out how to use that... – Crematorium