Why cpython exposes 'PyTuple_SetItem' as C-API if tuple is immutable by design?

Question

Tuple in python is immutable by design, so if we try to mutate a tuple object, python emits following TypeError which make sense.

>>> a = (1, 2, 3)
>>> a[0] = 12
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

So my question is, if tuple is immutable by design why cpython exposes PyTuple_SetItem as C-API?.

From the documentation it's described as

int PyTuple_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)

Insert a reference to object o at position pos of the tuple pointed to by p. Return 0 on success. If pos is out of bounds, return -1 and set an IndexError exception.

Isn't this statement exactly equal to tuple[index] = value in python layer?. If the goal was to create a tuple from collection of items we could have use PyTuple_Pack.

Additional note:

After lot of trial and error with ctypes.pythonapi I managed to mutate tuple object using PyTuple_SetItem

import ctypes

from ctypes import py_object

my_tuple = (1, 2, 3)
newObj = py_object(my_tuple)

m = "hello"

# I don't know why I need to Py_DecRef here. 
# Although to reproduce this in your system,  no of times you have 
# to do `Py_DecRef` depends on no of ref count of `newObj` in your system.
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)
ctypes.pythonapi.Py_DecRef(newObj)

ctypes.pythonapi.Py_IncRef(m)



PyTuple_SetItem = ctypes.pythonapi.PyTuple_SetItem
PyTuple_SetItem.argtypes = ctypes.py_object, ctypes.c_size_t, ctypes.py_object

PyTuple_SetItem(newObj, 0, m)
print(my_tuple) # this will print `('hello', 2, 3)`

Twigg

Related: Building self-referencing tuples – Strumpet

PyTuple_Pack is only usable in cases where the number of tuple elements is fixed. The only way to initialize a variable-sized tuple at the C level would be to call PyTuple_SetItem in a loop. – Inman

Answer 1

Similarly, there is a PyTuple_Resize function with the warning

Because tuples are supposed to be immutable, this should only be used if there is only one reference to the object. Do not use this if the tuple may already be known to some other part of the code. The tuple will always grow or shrink at the end. Think of this as destroying the old tuple and creating a new one, only more efficiently.

Looking at the source, there is a guard on the function

if (!PyTuple_Check(op) || Py_REFCNT(op) != 1) {
    .... error ....

Sure enough, this is only allowed when there is only 1 reference to the tuple - that reference being the thing that thinks its a good idea to change it. So, a tuple is "mostly immutable" but C code can change it in limited circumstances to avoid the penalty of creating a new tuple.

Sneaky

Answer 2

There is a note in the documentation that say:

Note This function “steals” a reference to o and discards a reference to an item already in the tuple at the affected position.

Otherwise in Cpython Github docs we can see more details about this function and especially how and why this function stealsthe object reference. We can read:

Incidentally, :c:func:PyTuple_SetItem is the only way to set tuple items; :c:func:PySequence_SetItem and :c:func:PyObject_SetItem refuse to do this since tuples are an immutable data type. You should only use :c:func:PyTuple_SetItem for tuples that you are creating yourself.

Equivalent code for populating a list can be written using :c:func:PyList_New and :c:func:PyList_SetItem.

However, in practice, you will rarely use these ways of creating and populating a tuple or list. There's a generic function, :c:func:Py_BuildValue, that can create most common objects from C values, directed by a :dfn:format string.

Innocent