How can I modify a Python traceback object when raising an exception?

Question

I'm working on a Python library used by third-party developers to write extensions for our core application.

I'd like to know if it's possible to modify the traceback when raising exceptions, so the last stack frame is the call to the library function in the developer's code, rather than the line in the library that raised the exception. There are also a few frames at the bottom of the stack containing references to functions used when first loading the code that I'd ideally like to remove too.

Thanks in advance for any advice!

Smite

Answer 1

What about not changing the traceback? The two things you request can both be done more easily in a different way.

1. If the exception from the library is caught in the developer's code and a new exception is raised instead, the original traceback will of course be tossed. This is how exceptions are generally handled... if you just allow the original exception to be raised but you munge it to remove all the "upper" frames, the actual exception won't make sense since the last line in the traceback would not itself be capable of raising the exception.
2. To strip out the last few frames, you can request that your tracebacks be shortened... things like traceback.print_exception() take a "limit" parameter which you could use to skip the last few entries.

That said, it should be quite possible to munge the tracebacks if you really need to... but where would you do it? If in some wrapper code at the very top level, then you could simply grab the traceback, take a slice to remove the parts you don't want, and then use functions in the "traceback" module to format/print as desired.

Froma

I downvoted, here's why: this answer offers good advice, but it doesn't actually answer the question. – Closemouthed

@BryanOakley: if advice is very relevant, and if it's too big to fit into a comment, it belongs in an answer. It's either that, or keep the knowledge away from SO, which would be very sad. So I don't think downvoting in these cases is appropriate. – Dewclaw

@max: I suppose you're right. I find myself answering questions like this as well -- "you can do that, but why.... here's something else to think about". Thanks for holding me accountable. I'd change my vote if I could. – Closemouthed

I'm okay with this answer offering an alternative, but I downvoted because I don't like that answer also doubts the necessity of doing the original thing -- I feel like a comment would be more appropriate for that. – Comeaux

With that said, it's perfectly reasonable to black-box certain parts of code that you would expect to run perfectly, so that you can focus more on the remaining traceback. Besides, if I'm allowed to catch and discard an entire exception, can't I discard just part of it? – Comeaux

Must disagree with the argument "[it] won't make sense since the last line in the traceback would not itself be capable of raising the exception". Take the line a += 1. It doesn't look "capable" of raising an exception (doesn't contain the word raise). But it will, if a is of type str - and the exception will make sense. Now suppose a is an instance of some custom class with an __iadd__, and that, just like str, it's nonsensical for the operand to be an int. With the same syntax, the traceback will now point to a deeper location rather than the line where the mistake is. – Semitone

@jez, with an 8 year old post I can't be 100% sure what I meant, but I suspect I could have better phrased it as merely "... would not appea capable of raising the exception". With your "a += 1", one would not expect to find, say, a KeyError being raised. If you got a traceback pointing to that line but with the lower levels (which did raise the KeyError) removed, troubleshooting would become much harder. Better to leave it (noisy though it is), showing explicitly how an apparently innocuous line like "a += 1" could have failed in that manner. But... 2009! :-) – Froma

Downvoting because I need to modify the stacktrace for valid reasons only to find this. Question: "how do you modify traceback". Answer: "you don't". I'd instead suggest a way to do it while at the same time strongly stressign you shouldn't. – Commonly

Horrible advice for cases, when we have complex graphs. – Poppycock

Answer 2

You can remove the top of the traceback easily with by raising with the tb_next element of the traceback:

except:
    ei = sys.exc_info()
    raise ei[0], ei[1], ei[2].tb_next

tb_next is a read_only attribute, so I don't know of a way to remove stuff from the bottom. You might be able to screw with the properties mechanism to allow access to the property, but I don't know how to do that.

Ddene

Any idea how to manage this in python 3? I can't get this to work myself. – Cylindrical

The equivalent of this in Python 3 would be: raise ei[0](ei[1]).with_traceback(ei[2].tb_next) – Cayla

That's not what he's asking. The causal chain was never mentioned. This guy wants to change, what is shown in the stack trace... put in some extra "virtual calls" etc. – Marathon

Answer 3

Take a look at what jinja2 does here:

https://github.com/mitsuhiko/jinja2/blob/5b498453b5898257b2287f14ef6c363799f1405a/jinja2/debug.py

It's ugly, but it seems to do what you need done. I won't copy-paste the example here because it's long.

Kaila

At the time that I write this, the above link gives a 404 error. – Closemouthed

Here's the current link: github.com/pallets/jinja/blob/master/src/jinja2/debug.py – Barbur

Answer 4

Starting with Python 3.7, you can instantiate a new traceback object and use the .with_traceback() method when throwing. Here's some demo code using either sys._getframe(1) (or a more robust alternative) that raises an AssertionError while making your debugger believe the error occurred in myassert(False): sys._getframe(1) omits the top stack frame.

What I should add is that while this looks fine in the debugger, the console behavior unveils what this is really doing:

Traceback (most recent call last):
  File ".\test.py", line 35, in <module>
    myassert_false()
  File ".\test.py", line 31, in myassert_false
    myassert(False)
  File ".\test.py", line 26, in myassert
    raise AssertionError().with_traceback(back_tb)
  File ".\test.py", line 31, in myassert_false
    myassert(False)
AssertionError

Rather than removing the top of the stack, I have added a duplicate of the second-to-last frame.

Anyway, I focus on how the debugger behaves, and it seems this one works correctly:

"""Modify traceback on exception.

See also https://github.com/python/cpython/commit/e46a8a
"""

import sys
import types


def myassert(condition):
    """Throw AssertionError with modified traceback if condition is False."""
    if condition:
        return

    # This function ... is not guaranteed to exist in all implementations of Python.
    # https://docs.python.org/3/library/sys.html#sys._getframe
    # back_frame = sys._getframe(1)
    try:
        raise AssertionError
    except AssertionError:
        traceback = sys.exc_info()[2]
        back_frame = traceback.tb_frame.f_back

    back_tb = types.TracebackType(tb_next=None,
                                  tb_frame=back_frame,
                                  tb_lasti=back_frame.f_lasti,
                                  tb_lineno=back_frame.f_lineno)
    raise AssertionError().with_traceback(back_tb)


def myassert_false():
    """Test myassert(). Debugger should point at the next line."""
    myassert(False)


if __name__ == "__main__":
    myassert_false()

Suction

excellent - this makes it possible to write a function decorator that removes itself from a stack trace when an exception occurs. – Gyrostabilizer

Answer 5

You might also be interested in PEP-3134, which is implemented in python 3 and allows you to tack one exception/traceback onto an upstream exception.

This isn't quite the same thing as modifying the traceback, but it would probably be the ideal way to convey the "short version" to library users while still having the "long version" available.

Cranston

Answer 6

For python3, here's my answer. Please read the comments for an explanation:

def pop_exception_traceback(exception,n=1):
    #Takes an exception, mutates it, then returns it
    #Often when writing my repl, tracebacks will contain an annoying level of function calls (including the 'exec' that ran the code)
    #This function pops 'n' levels off of the stack trace generated by exception
    #For example, if print_stack_trace(exception) originally printed:
    #   Traceback (most recent call last):
    #   File "<string>", line 2, in <module>
    #   File "<string>", line 2, in f
    #   File "<string>", line 2, in g
    #   File "<string>", line 2, in h
    #   File "<string>", line 2, in j
    #   File "<string>", line 2, in k
    #Then print_stack_trace(pop_exception_traceback(exception),3) would print: 
    #   File "<string>", line 2, in <module>
    #   File "<string>", line 2, in j
    #   File "<string>", line 2, in k
    #(It popped the first 3 levels, aka f g and h off the traceback)
    for _ in range(n):
        exception.__traceback__=exception.__traceback__.tb_next
    return exception

Barbi

How do you use this? Before raising, an exception's __traceback is None, and after raising, I see no way for the raising code to modify it. – Suction

Answer 7

This code might be of interest for you.

It takes a traceback and removes the first file, which should not be shown. Then it simulates the Python behavior:

Traceback (most recent call last):

will only be shown if the traceback contains more than one file. This looks exactly as if my extra frame was not there.

Here my code, assuming there is a string text:

try:
    exec(text)
except:
    # we want to format the exception as if no frame was on top.
    exp, val, tb = sys.exc_info()
    listing = traceback.format_exception(exp, val, tb)
    # remove the entry for the first frame
    del listing[1]
    files = [line for line in listing if line.startswith("  File")]
    if len(files) == 1:
        # only one file, remove the header.
        del listing[0]
    print("".join(listing), file=sys.stderr)
    sys.exit(1)

Trixi

The focus here is on 'simulating' - this does not change the traceback of the exception thrown (by sys.exit). – Suction

Yes, but it shows how to modify tracebacks. A little thinking of your own does not hurt ;-) – Trixi

Quite a number of people have invested quite a bit of thinking into that, and I have not found a working solution yet. So please don't imply that your answer + "a little thinking" will easily answer the OP's question. – Suction

The accepted answer was about "not changing the traceback". My answer assumed that also, and then this is simply some fiddling with a list of strings, which is easy to do instead of traceback modification. – Trixi

Yes, the accepted answer did state that. But both the question title and the text expressedly state the opposite, wanting to "modify the traceback". – Suction