How does PyArg_ParseTupleAndKeywords work?

Question

I've been trying to learn how to write C-extensions for Python and want to be sure I understand how PyArg_ParseTupleAndKeywords works.

I believe that the first argument is a PyObject pointer that points to an array of the arguments being passed into the C-extension function in the order they were passed. The second argument is a list of keywords that were passed, the positions at which they were passed and, very likely, some sort of indicator flag telling at which position the keywords begin and position becomes irrelevant.

PyArg_ParseTupleAndKeywords then uses its list of keywords (4th argument) to map between arguments specified with a keyword and both the format string (3rd argument) and addresses of C variables (5th & + arguments) to which the appropriate values should be copied.

Is my understanding correct? When I read through the online documentation, all I see are references to "positional arguments and keyword arguments", which leave me feeling a little bit in the dark. Where is the file for the Python interpreter that handles PyArg_ParseTupleAndKeywords?

Scull

Answer 1

Have you read through the opening explanation at http://docs.python.org/c-api/arg.html ? It does a pretty good job of explaining what's going on. Don't go right to the specific reference for PyArg_ParseTupleAndKeywords; it assumes you read the text above, and isn't very helpful by itself.

You've almost got it, though. The first argument is indeed a list of incoming positional arguments. The second is a map of incoming keyword arguments (mapping given keyword name to given value). The fourth argument is actually the list of keywords which your function is prepared to accept. Yes, the 3rd argument is the format string and the 5th and later are C pointers into which the values are copied.

You'll find PyArg_ParseTupleAndKeywords() under Python/getargs.c.

Servais

Rather than just the C API doc, there is a good readable summary at docs.python.org/2/extending/extending.html – Orthognathous

The documentation at docs.python.org/c-api/arg.html is pretty bad. – Lenrow

Answer 2

To emulate the following in python:

def keywords(a, b, foo=None, bar=None, baz=None):
    pass

The following will work:

static PyObject *keywords(PyObject *self, PyObject *args, PyObject *kwargs)
{
    char *a;
    char *b;
    char *foo = NULL;
    char *bar = NULL;
    char *baz = NULL;

    // Note how "a" and "b" are included in this
    // even though they aren't supposed to be in kwargs like in python
    static char *kwlist[] = {"a", "b", "foo", "bar", "baz", NULL};

    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ss|sss", kwlist, 
                                     &a, &b, &foo, &bar, &baz)) 
    {
        return NULL;
    }

    printf("a is %s\n", a);
    printf("b is %s\n", b);
    printf("foo is %s\n", foo);
    printf("bar is %s\n", bar);
    printf("baz is %s\n", baz);

    Py_RETURN_NONE;
}

// ...

static PyMethodDef SpamMethods[] = 
{
    // ...
    {"keywords", (PyCFunction) keywords, METH_VARARGS | METH_KEYWORDS, "practice kwargs"},
    {NULL, NULL, 0, NULL}
    // ...
}

And to use it:

from spam import keywords

keywords()         # Fails, require a and b
keywords('a')      # fails, requires b
keywords('a', 'b')
keywords('a', 'b', foo='foo', bar='bar', baz='baz') 
keywords('a', 'b','foo', 'bar', 'baz')
keywords(a='a', b='b', foo='foo', bar='bar', baz='baz')

Tableland

Weird— I can't edit. When you cast the "keywords" function, you want to cast it as PyCFunctionWithKeywords, rather than PyCFunction. – Willawillabella

@JimPivarski I seems like it would have to be PyCFunction #10264580 – Alena

Setting language = "c++" on setup.py and build it with -std=c++11 throws this warning warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings] for the line static char *kwlist[] = {"", "", NULL}; – Bice

C++ has some different rules around const. In this case, use const char *kwlist[] instead of static char *kwlist. – Servais

@user, try const_cast<char **>(kwlist), – Juanitajuanne

I have a compile error "cannot initialize a member subobject of type 'PyCFunction' : different number of parameters (2 vs 3)" and I do PyObject* set_opt_filter_values(PyObject *self, PyObject *args, PyObject *kwargs) and {"set_opt_filter_values", set_opt_filter_values, METH_VARARGS | METH_KEYWORDS, "sss"} – Schoolhouse