Python inspect.stack is slow

Question

I was just profiling my Python program to see why it seemed to be rather slow. I discovered that the majority of its running time was spent in the inspect.stack() method (for outputting debug messages with modules and line numbers), at 0.005 seconds per call. This seems rather high; is inspect.stack really this slow, or could something be wrong with my program?

Luftwaffe

Answer 1

inspect.stack() does two things:

• collect the stack by asking the interpreter for the stack frame from the caller (sys._getframe(1)) then following all the .f_back references. This is cheap.

• per frame, collect the filename, linenumber, and source file context (the source file line plus some extra lines around it if requested). The latter requires reading the source file for each stack frame. This is the expensive step.

To switch off the file context loading, set the context parameter to 0:

inspect.stack(0)

Even with context set to 0, you still incur some filesystem access per frame as the filename is determined and verified to exist for each frame.

Hopper

That cut down the time spent in the inspect.stack method by a factor of about 5. From there I think I'll just work to reduce the number of debug messages logged (or combine them). Thanks for the info. – Luftwaffe

Answer 2

inspect.stack(0) can be faster than inspect.stack(). Even so, it is fastest to avoid calling it altogether, and perhaps use a pattern such as this instead:

frame = inspect.currentframe()
while frame:
    if has_what_i_want(frame):  # customize
        return what_i_want(frame)  # customize
    frame = frame.f_back

Note that the last frame.f_back is None, and the loop will then end.

sys._getframe(1) should obviously not be used because it is an internal method.

As an alternative, inspect.getouterframes(inspect.currentframe()) can be looped over, but this is expected to be slower than the above approach.

Party

Answer 3

Here's a concrete example building on the other answers, showing how to efficiently walk back up the stack to find the typical caller information (filename, line number, function name) incorporated into debug messages.

import sys
from collections import namedtuple


FrameInfo = namedtuple('FrameInfo', ['filename', 'lineno', 'function'])


def frame_info(walkback=0):
    # NOTE: sys._getframe() is a tiny bit faster than inspect.currentframe()
    #   Although the function name is prefixed with an underscore, it is
    #   documented and fine to use assuming we are running under CPython:
    #
    #   https://docs.python.org/3/library/sys.html#sys._getframe
    #
    frame = sys._getframe().f_back

    for __ in range(walkback):
        f_back = frame.f_back
        if not f_back:
            break

        frame = f_back

    return FrameInfo(frame.f_code.co_filename, frame.f_lineno, frame.f_code.co_name)

Vanda