Why can't I pickle an error's Traceback in Python?

Question

I've since found a work around, but still want to know the answer.

Aldebaran

Then you should explain what exactly you are trying to do. You can convert your traceback to a string using traceback.format_exec(). Strings are eminently pickleable. – Vonvona

I'm not necessarily complaining, but I am interested in why this was down-voted exactly. Out of 1) lack of research, 2) unclear, or 3) not useful, I guess I'd chose (1) if I had to, but does that mean I should list all the Google results and books that didn't contain the answer I was after? – Aldebaran

@Aldebaran I haven't downvoted, but if I had to guess at the provenance of the three -1s, I'd say it was due to the lack of detail in your question. Why are you trying to pickle a traceback? What was the work-around you found? – Anticlinorium

What work around did you find? – Monospermous

There is a library called eliot that does a very nice job at logging tracebacks for you. Here's how eliot does traceback logging: github.com/itamarst/eliot/blob/master/eliot/_traceback.py#L82 – Parmer

related: #142302 – Calaverite

Answer 1

The traceback holds references to the stack frames of each function/method that was called on the current thread, from the topmost-frame on down to the point where the error was raised. Each stack frame also holds references to the local and global variables in effect at the time each function in the stack was called.

Since there is no way for pickle to know what to serialize and what to ignore, if you were able to pickle a traceback you'd end up pickling a moving snapshot of the entire application state: as pickle runs, other threads may be modifying the values of shared variables.

One solution is to create a picklable object to walk the traceback and extract only the information you need to save.

Peppermint

Thanks for the super clear answer.. makes total sense. My dream of shipping LogRecord transparently across the network is shattered, but now I understand why it can't be done... – Pentaprism

Answer 2

You can use tblib

    try:
        1 / 0
    except Exception as e:
         raise Exception("foo") from e
except Exception as e:
    s = pickle.dumps(e)
raise pickle.loads(s)

Divestiture

Answer 3

I guess you are interested in saving the complete call context (traceback + globals + locals of each frame).

That would be very useful to determine a difference of behavior of the same function in two different call contexts, or to build your own advanced tools to process, show or compare those tracebacks.

The problem is that pickl doesn't know how to serialize all type of objects that could be in the locals or globals.

I guess you can build your own object and save it, filtering out all those objects that are not picklabe. This code can serve as basis:

import sys, traceback

def print_exc_plus():
    """
    Print the usual traceback information, followed by a listing of all the
    local variables in each frame.
    """
    tb = sys.exc_info()[2]
    while 1:
        if not tb.tb_next:
            break
        tb = tb.tb_next
    stack = []
    f = tb.tb_frame
    while f:
        stack.append(f)
        f = f.f_back
    stack.reverse()
    traceback.print_exc()
    print "Locals by frame, innermost last"
    for frame in stack:
        print
        print "Frame %s in %s at line %s" % (frame.f_code.co_name,
                                             frame.f_code.co_filename,
                                             frame.f_lineno)
        for key, value in frame.f_locals.items():
            print "\t%20s = " % key,
            #We have to be careful not to cause a new error in our error
            #printer! Calling str() on an unknown object could cause an
            #error we don't want.
            try:                   
                print value
            except:
                print "<ERROR WHILE PRINTING VALUE>"

but instead of printing the objects you can add them to a list with your own pickable representation ( a json or yml format might be better).

Maybe you want to load all this call context in order to reproduce the same situation for your function without run the complicated workflow that generate it. I don't know if this can be done (because of memory references), but in that case you would need to de-serialize it from your format.

Roster

More here: code.activestate.com/recipes/… – Roster