Undefined behavior in Python

Question

What undefined behavior does Python have? Undefined meaning not in the specification of the language. The only example I know of is modifying a sequence while iterating through it. Before learning about that I used to think that Python didn't have undefined behavior, so I assume it is extremely rare.

Doorstep

How would you define "undefined" in your context? – Coinstantaneous

Any behavior that is not explicitly defined in the language spec is undefined. – Dziggetai

Note, modifying a sequence while iterating over it is entirely predictable. Whether it's defined or not depends on if sequence iteration is explicitly defined somewhere in the spec, which I don't think it is. – Kotick

@Kotick That's really interesting! Do you know where I would look to learn about how it works? I assume its implementation specific. – Doorstep

@Doorstep for built-in sequences, it essentially keeps a counter, returns self[i], then increments i until it reaches len(self). So, here's an answer to another question where I give a Python implementation – Kotick

Answer 1

Here are two:

• Garbage collection
• Ordered dictionaries in Python 3.6

---

I would argue that garbage collection falls into this category. The main Python implementation (e.g. CPython) uses reference counting for garbage collection, but this is not mandated by the language spec.

Further, you cannot rely on a variable being garbage collected (and therefore "cleaned up") at any particular point in Python.

Consider this:

f = open("file.txt").read()

This is pretty common in new-to-python code, and seems safe enough, but there is no explicit .close() call on the opened file object. When the object is destroyed, .close() is called implicitly. Because of the way CPython works this almost always gets destroyed in a timely manner and this ends up being OK in practice. But for other python interpreters this many not be the case and you could end up with dangling file objects. This is why you always see folks saying it is safer to do

with open("file.txt") as fl:
   f = fl.read()

because now the closing of the object is not only explicit, but is guaranteed to happen at a particular line of code.

---

In Python 3.6, dictionaries were ordered as in implementation detail in CPython. It was not mandated by the spec, but people started to rely on this undefined behavior which would cause problems for users of other python interpreters, so in Python 3.7 they made ordered dictionaries part of the spec.

Gabriellegabrielli

Answer 2

Searching Python's documentation for the word "undefined" returns quite a few matches. Some of them more or less correspond to what would have been undefined behavior in the sense the C and C++ standards give to it.

In a few cases, different versions of the language change the meaning of what previously have been undefined behavior, by either legalizing a de-facto behavior exhibited in the past implementations, or requiring an exception to be thrown.

In many cases, undefined behavior creeps in where Python relies on the underlying host functionality without setting any guards for illegal cases.

All things involving concurrent/parallel/asynchronous programming are a rich source of cases for undefined behavior.

Below are a few examples I have found, in addition to what have already been mentioned in other answers.

1. Iterating over mutated list should be classified as either implementation defined or undefined behavior, I am not sure.

CPython implementation detail: While a list is being sorted, the effect of attempting to mutate, or even inspect, the list is undefined. The C implementation of Python makes the list appear empty for the duration, and raises ValueError if it can detect that the list has been mutated during a sort.

2. Exact semantics of __del__() for resurrected objects is also implementation-defined, bordering on undefined.

It is implementation-dependent whether __del__() is called a second time when a resurrected object is about to be destroyed

3. Identity of immutable types:

for immutable types, operations that compute new values may actually return a reference to any existing object with the same type and value, while for mutable objects this is not allowed. E.g., after a = 1; b = 1, a and b may or may not refer to the same object with the value one, depending on the implementation.

4. float type implementation is said to rely on underlying hardware's FPU. So in a way the behavior it will give is not fully defined the same way it is done for e.g. int.

5. concurrent.futures documentation gives 2 statement about undefined behavior:

add_done_callback(fn). If the callable raises a BaseException subclass, the behavior is undefined.

Changed in version 3.3: When one of the worker processes terminates abruptly, a BrokenProcessPool error is now raised. Previously, behaviour was undefined but operations on the executor or its futures would often freeze or deadlock.

6. email.parser mentions "undefined".

It is undefined what happens if feed() is called after this method has been called.

7. Function annotation syntax does not assign any semantics to them.

Function annotation syntax has been a Python feature since version 3.0 (PEP 3107), however the semantics of annotations has been left undefined.

8. time:

Warning. Passing an invalid or expired thread_id may result in undefined behavior, such as segmentation fault.

Slagle