How to deal with uint8_t on a Python Extension?

Question

I would like to pass as argument of a function in my C module an array of uint8_t's.

I couldn't find a method to directly parse this array, so I'm parsing it to a PyObject_t and then iterating as a PyTuple_t object. This way, I need to cast each element PyObject_t of this tuple to uint8_t.

How can I do that, once that there is no PyInt_FromUINT8_t function or anything like it?

Armilla

Answer 1

You can usually just get away with B using unsigned char. According to Parsing Arguments you should just be able to do:

uint8_t b;
if (!PyArg_ParseTuple("b", &b)) {
    return NULL;
}

If not directly using arguments (e.g. you are dealing with a PyObject, simply use one of the PyInt_*, PyLong_* or PyNumber_* functions (https://docs.python.org/3/c-api/number.html?highlight=pynumber#c.PyNumber_AsSsize_t).

Converting from a uin8_t to a PyObject is simple as well, you can use PyInt_FromLong or PyLong_FromLong

Carmeliacarmelina

I'm working with arrays. In this case, iterate over a PyTuple or PyList using PyInt_FromLong won't cause an invalid memory access? This function will try to read 4 or 8 bytes but each element of the list has only 1 byte allocated. – Armilla

None of the PyXXX functions use pointers to integers, but simply return them by value; so PyInt_FromLong for example would accept an int as an argument, and it would be subject to normal type conversions and integer coercions. Can you explain a bit more what you're trying to do? – Carmeliacarmelina

@PedroAlves: A tuple cannot have uint8_ts in it. Where do you expect an array of uint8_ts to even come from on the Python side, anyway? You may want to use a bytes or bytearray object. – Dierdre

Indeed. The only way to get an actual array of uint8_t (such that you would not need to do manual conversion each time) is a bytearray or bytes object (bytes is slightly more efficient because its contents is allocated at the tail of the PyObject whereas the bytearray is mutable and requires a separate allocation. There is also "memory view" stuff which allows you to write adapters to expose iterators and bytes from your own objects, but perhaps more trouble than it's worth for you – Carmeliacarmelina

What you mean with "...bytes is slightly more efficient because its contents is allocated at the tail of the PyObject..."? Can you write a simple example that receives an array of bytes in a C extension method? – Armilla

I mean from the Python end. PyBytesObject { ... size_t n; char alloc[1] } vs PyBytearrayObject { ... size_t n; char **buf} – Carmeliacarmelina