Why does Python have a limit on the number of static blocks that can be nested?

Question

The number of statically nested blocks in Python is limited to 20. That is, nesting 19 for loops will be fine (although excessively time consuming; O(n^19) is insane), but nesting 20 will fail with:

SyntaxError: too many statically nested blocks

What is the underlying reason for having such a limit? Is there a way to increase the limit?

Oyler

Why do you want to increase this? Is this a practical issue for you? – Slot

I am still wondering about the relevance of this in real life. I would assume that hitting this limit is only possible with insane spaghetti code. Or what am I missing here? – Graiggrail

@Graiggrail Two words: autogenerated code. – Encincture

@Jason The same limit should apply to nested ifs. 20 nested ifs are a less contrived example than 20 nested fors. The "excessively time consuming" argument does not apply to them. – Encincture

Answer 1

This limit not only applies to for loops, but to all other control flow blocks as well. The limit for the number of nested control flow blocks is defined inside of code.h with a constant named CO_MAXBLOCKS:

#define CO_MAXBLOCKS 20 /* Max static block nesting within a function */

This constant is used to set the maximum size for the stack Python uses to execute exceptions and loops named blockstack. This limit is imposed upon all frame objects and is shown in frameobject.h:

int blockstack[CO_MAXBLOCKS];       /* Walking the 'finally' blocks */

The most likely reason for this limit is to keep memory usage at a sane level when executing nested blocks. It's probably similar to the limit Python imposes on recursive calls. This limit can be seen being enforced in compile.c:

if (c->u->u_nfblocks >= CO_MAXBLOCKS) {
    PyErr_SetString(PyExc_SyntaxError,
                    "too many statically nested blocks");
    return 0;
}

A more concrete answer as to why Python has this specfic limit and why they cannot get rid of it, was given by Michael Hudson in a 2004 Python mailing list letter:

Spot on. This has to do with the 'blockstack', very much an internal detail to Python's implementation. We'd like to get rid of it (not because we want to let people write code with more than 20 nested for loops :-) but it's not especially easy (finally: blocks are the biggest problem).

Note that in Python 2.6 and lower, breaking the maximum number of nested loops would've cause a SystemError not a SyntaxError. This was changed however in Python 3 and back-patched to Python 2.7 so a SyntaxError would be raised instead. This was documented in #issue 27514:

Issue #27514: Make having too many statically nested blocks a SyntaxError instead of SystemError.

The reason for this change in exception types was given by Serhiy Storchaka :

[...] SystemError is not an exception that should be raised. SystemError is for errors that can't be occurred in normal case. It should only be caused by incorrect use of C API or hacking Python internals. I think SyntaxError is more appropriate in this case [...].

Reeta

So the solution (which no-one is recommending) for the problem (that no-one actually has) is to hack the CPython source code? – Slot

@Slot Yup, I guess so. If you change CO_MAXBLOCKS then recompile, I guess you could have more than twenty nested blocks in theory. – Reeta

@cᴏʟᴅsᴘᴇᴇᴅ I didn't see that, but the test suites would likely complain too github.com/python/cpython/blob/… and github.com/python/cpython/blob/… – Slot

@Slot I do believe you can make build without make tests, if I'm not mistaken. – Durstin

@Rightleg Thanks. I had fun writing this answer. To be honest, I never knew Python had this limit. I learned something new as well. – Reeta

Answer 2

This has to do with the blockstack, which is a stack of the byte code addresses, and is used to execute code blocks such as loops and exceptions.

It just so happens that a version of C (older than C99) had set this limit to 20, and since the CPython interpreter is built with C, the same convention has been followed:

#define CO_MAXBLOCKS 20 /* Max static block nesting within a function */

The constant 20 seems to be set out of convention, and nothing more.

[Links courtesy Christian Dean.]

---

Why is the limit 20?

If the argument of convention isn't convincing, then take a look at The Zen of Python:

In [4]: import this
The Zen of Python, by Tim Peters

...
Flat is better than nested.
...

---

How can you increase this value?

Since this value is a hardcoded constant, the only way to change it to effect in your programs is to rebuild your python distribution and run your script on the new build.

1. Download the cpython source code from github

2. Navigate to cpython/Include/code.h

3. Change the value of CO_MAXBLOCKS to anything greater than 20

4. Recompile Python (disable tests, they'll complain)

Durstin

You know, C99 (The one standard I found just now) allows a minimum of 127 levels of nesting. That's a bit more than 20. And GCC for example goes for "whatever we have the resources to compile" anyway. – Demello

@Demello Yep, I did see that was the limit for C99. This is more of a convention than out of any imposed restriction except memory limits. – Durstin

Actually the number 20 makes some sense to me, it's not completely random: the recommended settings are 4 spaces for indentation, and 80 characters per line. And 80/4 = 20. – Encincture

ANSI C doesn't set the nesting limit to 20, either. It is 15 for "compound statements, iteration control structures, and selection control structures". So, the level of 20 had to be chosen for a later C standard that had a higher limit, and it was (as you already stated) chosen rather arbitrarily. – Zetana

@CodyGray Thanks for clarifying that. I never really found any official source for that information, just discussions in various forums. – Durstin

Answer 3

See answer here: too many statically nested blocks python You cannot increase it as it is builtin to the python syntax. The limit applies to any kind of code stack (exceptions, loops, etc.) and is a decision by the designers (presumably to keep memory usage reasonable). One weird thing is that here: https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Include/code.h#L95 it says 20 is the max number in a function. But I just tried nesting 23 for loops, not inside a function, and you still get the error.

Outdistance

I suppose that in a function just means that if you call another function at level 20, that one still can nest 20 deep. – Negligee

Oh, that makes sense – Outdistance