How to create a traceback object

Question

I want to create a traceback like the one returned by sys.exc_info()[2]. I don't want a list of lines, I want an actual traceback object:

<traceback object at 0x7f6575c37e48>

How can I do this? My goal is to have it include the current stack minus one frame, so it looks the the caller is the most recent call.

Of

What do you want this traceback for? The result of traceback.extract_stack and/or inspect.stack isn't a traceback object, but it's not just a list of lines, either. Is one of those sufficient? – Rushing

3rd party library only accepts a traceback object and I didn't feel like hacking it. – Of

Does "only accepts a traceback object" mean "checks isinstance(t, types.TracebackType)? If so… well, I wouldn't be surprised if traceback is one of the handful of types that won't allow itself to be used as a base, but I'd at least check first. If not, what does it mean? – Rushing

Answer 1

There's no documented way to create traceback objects.

None of the functions in the traceback module create them. You can of course access the type as types.TracebackType, but if you call its constructor you just get a TypeError: cannot create 'traceback' instances.

The reason for this is that tracebacks contain references to internals that you can't actually access or generate from within Python.

---

However, you can access stack frames, and everything else you'd need to simulate a traceback is trivial. You can even write a class that has tb_frame, tb_lasti, tb_lineno, and tb_next attributes (using the info you can get from traceback.extract_stack and one of the inspect functions), which will look exactly like a traceback to any pure-Python code.

So there's a good chance that whatever you really want to do is doable, even though what you're asking for is not.

Rushing

Jinja2 does this hack plus some more complex things. github.com/mitsuhiko/jinja2/blob/… – Moncada

@Moncada There's some nice stuff there. In particular, dynamically proxying to a real traceback while adding extra functionality is an obvious good idea for many use cases; I should have thought of that... – Rushing

It's also useful when building debuggers in pytest, so you can filter out __tracebackhide__ frames. – Opia

Answer 2

Since Python 3.7 you can create traceback objects dynamically from Python.
To create traceback identical to one created by raise:

raise Exception()

use this:

import sys
import types

def exception_with_traceback(message):
    tb = None
    depth = 0
    while True:
        try:
            frame = sys._getframe(depth)
            depth += 1
        except ValueError as exc:
            break

        tb = types.TracebackType(tb, frame, frame.f_lasti, frame.f_lineno)

    return Exception(message).with_traceback(tb)

Relevant documentation is here:

• https://docs.python.org/3/library/types.html#types.TracebackType
• https://docs.python.org/3/reference/datamodel.html#traceback-objects
• https://docs.python.org/3/library/sys.html#sys._getframe

Extemporaneous

This is excellent, thank you. – Nosedive

Answer 3

If you really need to fool another library—especially one written in C and using the non-public API—there are two potential ways to get a real traceback object. I haven't gotten either one to work reliably. Also, both are CPython-specific, require not just using the C API layer but using undocumented types and functions that could change at any moment, and offer the potential for new and exciting opportunities to segfault your interpreter. But if you want to try, they may be useful for a start.

---

The PyTraceBack type is not part of the public API. But (except for being defined in the Python directory instead of the Object directory) it's built as a C API type, just not documented. So, if you look at traceback.h and traceback.c for your Python version, you'll see that… well, there's no PyTraceBack_New, but there is a PyTraceBack_Here that constructs a new traceback and swaps it into the current exception info. I'm not sure it's valid to call this unless there's a current exception, and if there is a current exception you might be screwing it up by mutating it like this, but with a bit of trial&crash or reading the code, hopefully you can get this to work:

import ctypes
import sys

ctypes.pythonapi.PyTraceBack_Here.argtypes = (ctypes.py_object,)
ctypes.pythonapi.PyTraceBack_Here.restype = ctypes.c_int

def _fake_tb():
    try:
        1/0
    except:
        frame = sys._getframe(2)
        if ctypes.pythonapi.PyTraceBack_Here(frame):
            raise RuntimeError('Oops, probably hosed the interpreter')
        raise

def get_tb():
    try:
        _fake_tb()
    except ZeroDivisionError as e:
       return e.__traceback__

---

As a fun alternative, we can try to mutate a traceback object on the fly. To get a traceback object, just raise and catch an exception:

try: 1/0
except exception as e: tb = e.__traceback__ # or sys.exc_info()[2]

The only problem is that it's pointing at your stack frame, not your caller's, right? If tracebacks were mutable, you could fix that easily:

tb.tb_lasti, tb.tb_lineno = tb.tb_frame.f_lasti, tb.tb_frame.f_lineno
tb.tb_frame = tb.tb_frame.f_back

And there's no methods for setting these things, either. Notice that it doesn't have a setattro, and its getattro works by building a __dict__ on the fly, so obviously the only way we're getting at this stuff is through the underlying struct. Which you should really build with ctypes.Structure, but as a quick hack:

p8 = ctypes.cast(id(tb), ctypes.POINTER(ctypes.c_ulong))
p4 = ctypes.cast(id(tb), ctypes.POINTER(ctypes.c_uint))

Now, for a normal 64-bit build of CPython, p8[:2] / p4[:4] are the normal object header, and after that come the traceback-specific fields, so p8[3] is the tb_frame, and p4[8] and p4[9] are the tb_lasti and tb_lineno, respectively. So:

p4[8], p4[9] = tb.tb_frame.f_lasti, tb.tb_frame.f_lineno

But the next part is a bit harder, because tb_frame isn't actually a PyObject *, it's just a raw struct _frame *, so off you go to frameobject.h, where you see that it really is a PyFrameObject * so you can just use the same trick again. Just remember to _ctypes.Py_INCREF the frame's next frame and Py_DECREF the frame itself after doing reassigning p8[3] to point at pf8[3], or as soon as you try to print the traceback you'll segfault and lose all the work you'd done writing this up. :)

Rushing

Answer 4

"In order to better support dynamic creation of stack traces, types.TracebackType can now be instantiated from Python code, and the tb_next attribute on tracebacks is now writable."

There is an explanation(in python 3.7) for the same in here(python 3.7) https://docs.python.org/3/library/types.html#types.TracebackType

Tito

Answer 5

As others have pointed out, it's not possible to create traceback objects. However, you can write your own class that has the same properties:

from collections import namedtuple
fake_tb = namedtuple('fake_tb', ('tb_frame', 'tb_lasti', 'tb_lineno', 'tb_next'))

You can still pass instances of this class to some Python functions. Most notably, traceback.print_exception(...), which produces the same output as Python's standard excepthook.

If you (like me) encountered this problem because you are working on a PyQt-based GUI app, you may also be interested in a more comprehensive solution laid out in this blog post.

Jerkwater