Extending python with C: Pass a list to PyArg_ParseTuple

Question

I have been trying to get to grips with extending python with C, and so far, based on the documentation, I have had reasonable success in writing small C functions and extending it with Python.

However, I am now struck on a rather simple problem - to which I am not able to find a solution. So, what I'd like to do is pass a double list to my C function. For example, to pass an int, I do the following:

int squared(int n)
{
    if (n > 0)
        return n*n;
    else
        return 0;
}

static PyObject*
squaredfunc(PyObject* self, PyObject* args)
{
    int n;

    if (!PyArg_ParseTuple(args, "i", &n))
        return NULL;

    return Py_BuildValue("i", squared(n));
}

This passes the int n with no problems to my C function named squared.

But, how does one pass a list to the C function? I did try to google it and read the docs, and so far, I havent found anything useful on this.

Would really appreciate if someone could point me in the right direction.

Thanks.

Amy

I think you need "O" rather than "i" to say that your accepting an arbitrary object. – Nisan

@mgilson: thanks for that. But, how do I then change my int n declaration to reflect this? Should I just be doing double n[]? – Amy

No, it would be a PyObject *lst;, or something similar I would think. – Nisan

@mgilson: I am wondering if there was an example somewhere of this where I can look at and try to understand? Thanks again. – Amy

Not sure. I don't know much, and what I do know is from reading the python source code/documentation you already linked. – Nisan

Answer 1

PyArg_ParseTuple can only handle simple C types, complex numbers, char *, PyStringObject *, PyUnicodeObject *, and PyObject *. The only way to work with a PyListObject is by using some variant of "O" and extracting the object as a PyObject *. You can then use the List Object API to check that the object is indeed a list (PyList_Check). Then you can then use PyList_Size and PyList_GetItem to iterate over the list. Please note that when iterating, you will get PyObject * and will have to use the floating point API to access the actual values (by doing PyFloat_Check and PyFloat_AsDouble.) As an alternative to the List API, you can be more flexible and use the iterator protocol (in which case you should just use PyIter_Check). This will allow you to iterate over anything that supports the iterator protocol, like lists, tuples, sets, etc.

Finally, if you really want your function to accept double n[] and you want to avoid all of that manual conversion, then you should use something like boost::python. The learning curve and APIs are more complex, but boost::python will handle all of the conversions for you automatically.

Here is an example of looping using the iterator protocol (this is untested and you'd need to fill in the error handling code):

PyObject *obj;

if (!PyArg_ParseTuple(args, "O", &obj)) {
  // error
}

PyObject *iter = PyObject_GetIter(obj);
if (!iter) {
  // error not iterator
}

while (true) {
  PyObject *next = PyIter_Next(iter);
  if (!next) {
    // nothing left in the iterator
    break;
  }

  if (!PyFloat_Check(next)) {
    // error, we were expecting a floating point value
  }

  double foo = PyFloat_AsDouble(next);
  // do something with foo
}

Clova

Answer 2

The PyArg_ParseTuple function allows you to cast directly to a Python object subtype using the format string "O!" (notice-this is different than just plain "O"). If the argument does not match the specified PyObject type, it will throw a TypeError. For example:

PyObject *pList;
PyObject *pItem;
Py_ssize_t n;
int i;

if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &pList)) {
    PyErr_SetString(PyExc_TypeError, "parameter must be a list.");
    return NULL;
}

n = PyList_Size(pList);
for (i=0; i<n; i++) {
    pItem = PyList_GetItem(pList, i);
    if(!PyInt_Check(pItem)) {
        PyErr_SetString(PyExc_TypeError, "list items must be integers.");
        return NULL;
    }
}

As a side note, remember that iterating over the list using PyList_GetItem returns a borrowed reference to each item, so you do not need Py_DECREF(item) to handle the reference count. On the other hand, with the useful Iterator Protocol (see the answer by @NathanBinkert), each item returned is a new reference - so you must remember to discard it when done using Py_DECREF(item).

Passerby

Is there a way to disable PyArg_ParseTuple from throwing an exception? I want to have a function that can take multiple types. – Sit

@Tony: The point of the "O!", &PyList_Type format string in PyArg_ParseTuple(args, "O!", &PyList_Type, &pList) is to make it check the type and throw TypeError. If you don't want it to check the type, then just use PyArg_ParseTuple(args, "O", &pList) instead. (Btw, separately, the & in &PyList_Type is super important! That bit me the first time.) – Abiotic

@Tony: Alternatively, if by "a function that can take multiple types" you meant "a function where some of the parameters are optional," that's handled by the "|" format specifier. If you still have questions in this area, I recommend asking your own separate SO question. – Abiotic