Return list of new custom-class objects in python C API

Question

I need to create a new list via the python C API containing new copies of objects of a Quaternion class I've written (in C++). [Actually, I'd really like a numpy array, but any sort of sequence would do.] But I'm getting seg faults with everything I've tried. I'm terrible with pointers, so that's not a big surprise. I'm thinking maybe I need to give python ownership of the objects I create with new.

The best I've gotten so far appears below. Am I not supposed to copy-construct the Quaternion while newing it? Am I doing something else stupid? Do I need to tell python it owns the reference now? Should the returned list exist and live a happy life, as I expected?

PyObject* Objectify(std::vector<Quaternion>& v) {
  Py_ssize_t size = v.size();
  PyArrayObject* list = (PyArrayObject*) PyList_New(size);
  for(Py_ssize_t i=0; i<size; ++i) {
    PyObject* o = (PyObject*) new Quaternion(v[i]);
    PyList_SET_ITEM((PyObject*)list, i, o);
  }
  return PyArray_Return(list);
}

I can verify that the list still has the correct elements just before the return. That is, after the loop above, I make a new loop and print out the original values next to the list values, and they match. The function will return, and I can keep using python. But once the list is used outside the loop, the segfaults happen.

[Actually, this is all being done in SWIG, and this code is found in a typemap with slightly different variable names, but I can look in the _wrap.cxx file, and see that it's just how I would have written it.]

Tirewoman

Answer 1

Now that I see it, it's obvious. I can't just cast an arbitrary type to a PyObject; it needs to actually have some actual python structure. E.g., I'm thinking a PyArrayObject can be cast as such, but my random class needs to be wrapped, as dastrobu explained on my other question.

As I mentioned, I'm using SWIG, which already wraps Quaternion into some kind of PyObject (though it's given some other name to denote that it's made by SWIG). So, while this doesn't answer my question in general, the following does exactly what I need (including creating numpy arrays, rather than just lists):

npy_intp size = v.size();
PyArrayObject *npy_arr = reinterpret_cast<PyArrayObject*>(PyArray_SimpleNew(1, &size, NPY_OBJECT));
PyObject** data = static_cast<PyObject**>(PyArray_DATA(npy_arr));
for(npy_intp i=0; i<size; ++i) {
  PyObject* qobj = SWIG_NewPointerObj((new Quaternions::Quaternion(v[i])),
                                      SWIGTYPE_p_Quaternions__Quaternion, SWIG_POINTER_OWN);
  if(!qobj) {SWIG_fail;}
  data[i] = qobj;
  Py_INCREF(qobj);
}

I should also point out that I originally tried using PyArray_SET_ITEM to assign the items of the list in this code, but kept getting segfaults, which is why I use this weird method. I wonder if it has something to do with the offsets of the NPY_OBJECT...

Anyway, I hope that helps someone else in the future.

Tirewoman