Correct way to handle Exception Stack Traces in Python

Question

I am writing some code which can generate Exceptions deep in the stack, and I have a layer nearer the top which catches these exceptions and sends them off for processing by an error handling module. What I would like is for the error handling module to be able to print the stack trace into its error log, but I'm finding it difficult to work out what the way to do this is.

Some notes on the background:

• The code is multi-threaded, so I'm not certain of the behaviour of sys.last_traceback

• I have tried capturing the stack in the constructor of the exception itself. sys.last_traceback is sometimes None in this case (since it only exists in the case of unhandled exceptions), and not always correct. I am currently toying with using

  self.stack = traceback.format_stack()[:-2]

in the constructor of the exception, and while this looks promising in terms of information, it doesn't feel like the "right" way to do it

• All examples of how to do this that I've been able to find show how to print the stack trace in the except block, not in some later error handling module, and this appears to be different in behaviour from what I want. (see, for example, Print current call stack from a method in Python code)

• I've mined the traceback module in the python docs (http://docs.python.org/library/traceback.html), and haven't been able to figure out whether this even does what I want. It seems mostly concerned with formatting tracebacks as you might retrieve from sys.last_traceback. It has some examples of usage, and none of them line up with what I'm trying to do.

I don't believe that I'm the first person to ever want to do this, so I must be missing something. Any pointers in the right direction much appreciated.

Blairblaire

"All examples of how to do this that I've been able to find show how to print the stack trace in the except block, not in some later error handling module, and this appears to be different in behaviour from what I want." I don't understand how. Show an example, and the expected and actual output. – Susannesusceptibility

Hi @KarlKnechtel - another example I found is doughellmann.com/PyMOTW/traceback The error is reported using print_exc() in the except block, but when I use this function outside an except block I get nothing (None) – Blairblaire

So... it only works inside an except block. But the entire point is that an exception is getting thrown, and then later on, there is "a layer nearer the top which catches these exceptions". So what's the problem with doing it inside that except block? – Susannesusceptibility

Answer 1

Your first line on handling the exception could be:

exc_type, exc_value, exc_traceback = sys.exc_info()

You can store these variables or pass them any how you like, then later use the traceback module to display them.

Koniology

Hi @ali-afshar - yes, I had wondered about using something like this, but the downside is that this needs to appear in every except block and then assign the variables to some properties of the exception, for example. This places a burden on the developer to remember to do this, and makes bugs easier to introduce. What would be better would be to have this line of code /inside/ the exception constructor, but that doesn't appear to work. – Blairblaire

@RichardJ: note that not all Exceptions are thrown in the place they are constructed, while this will be true for most exceptions, it is strictly speaking incorrect to assume so. – Henrik

"the downside is that this needs to appear in every except block"... but I thought the entire point of your design is that you're letting all the exceptions fall through to a single except block higher up! – Susannesusceptibility

The whole thing is pretty broken. – Koniology

Answer 2

How about adding some decorator in the functions that you want to protect? For example:

@onerrorignore
def foo(a, b, c):
    ...

the decorator would look something like:

def onerrorignore(f):
    def _f(*args, **kwargs):
        try:
            ret = f()
        except:
            error_handler.handle(sys.exc_info())
        else:
            return ret
    return _f

Henrik

Answer 3

If I understand you right, you want to re-throw an exception while keeping the stacktrace? So you've got a call hierarchy like this:

error_handling_module # you want to print the stacktrace here
|
something_else
|
module_excepting_and_rethrowing
|
something_else
|
module_raising_an_error

In you module_excepting_and_rethrowing you can do:

except Exception:
   exc_type, exc_value, exc_traceback = sys.exc_info()
   raise NewException, exc_value, exc_traceback

In Python3 you can also do:

 except Exception as e:
     raise NewException from e

Misconstrue