Exception traceback is hidden if not re-raised immediately

Question

I've got a piece of code similar to this:

import sys

def func1():
    func2()

def func2():
    raise Exception('test error')

def main():
    err = None

    try:
        func1()
    except:
        err = sys.exc_info()[1]
        pass

    # some extra processing, involving checking err details (if err is not None)

    # need to re-raise err so caller can do its own handling
    if err:
        raise err

if __name__ == '__main__':
    main()

When func2 raises an exception I receive the following traceback:

Traceback (most recent call last):
  File "err_test.py", line 25, in <module>
    main()
  File "err_test.py", line 22, in main
    raise err
Exception: test error

From here I don't see where the exception is coming from. The original traceback is lost.

How can I preserve original traceback and re-raise it? I want to see something similar to this:

Traceback (most recent call last):
  File "err_test.py", line 26, in <module>
    main()
  File "err_test.py", line 13, in main
    func1()
  File "err_test.py", line 4, in func1
    func2()
  File "err_test.py", line 7, in func2
    raise Exception('test error')
Exception: test error

Aalesund

Answer 1

A blank raise raises the last exception.

# need to re-raise err so caller can do its own handling
if err:
    raise

If you use raise something Python has no way of knowing if something was an exception just caught before, or a new exception with a new stack trace. That's why there is the blank raise that preserves the stack trace.

Reference here

Materially

Worth mentioning that this doesn't work in Python 3. – Kowalewski

"This" in yprez's comment means "empty raise after leaving the except block." The bare "raise" does work in Python 3 (but only inside of the except block.) – Apace

Answer 2

It is possible to modify and rethrow an exception:

If no expressions are present, raise re-raises the last exception that was active in the current scope. If no exception is active in the current scope, a TypeError exception is raised indicating that this is an error (if running under IDLE, a Queue.Empty exception is raised instead).

Otherwise, raise evaluates the expressions to get three objects, using None as the value of omitted expressions. The first two objects are used to determine the type and value of the exception.

If a third object is present and not None, it must be a traceback object (see section The standard type hierarchy), and it is substituted instead of the current location as the place where the exception occurred. If the third object is present and not a traceback object or None, a TypeError exception is raised.

The three-expression form of raise is useful to re-raise an exception transparently in an except clause, but raise with no expressions should be preferred if the exception to be re-raised was the most recently active exception in the current scope.

So if you want to modify the exception and rethrow it, you can do this:

try:
    buggy_code_which_throws_exception()
except Exception as e:
    raise Exception, "The code is buggy: %s" % e, sys.exc_info()[2]

Soave

Interesting. The accepted answer handles the OP's use case better than this, but this is interesting as a more general answer. I can't see much use for it though since the traceback gets really misleading if you catch, say, a ValueError and raise a RuntimeError (you can't see, then, that a ValueError was ever involved), and the only cases I've come across personally when I've wanted to preserve the traceback but do something more complex than just raise with no arguments were cases where I wanted to raise an exception of a different type. – Poussette

I've used this to rethrow the same exception with a different message, including more details about the conditions that caused the exception, which are available in the outer scope but not the inner. – Soave

There are sometimes very good reasons for using this three expression form of raise. Your answer just helped me write a decorator that wraps integration tests and on failure takes a screenshot. And then raises the original assertion failure. The trouble being that the traceback was getting clobbered by try/excepts in the screen shot taking code. So thanks! – Mulvey

is there a way to do this in a Python 2 and Python 3 compatible way? I get SyntaxError in Python 3. – Ossie

@elias six.reraise(exc_type, exc_value, exc_traceback=None) – Soave

Answer 3

You can get a lot of information about the exception via the sys.exc_info() along with the traceback module

try the following extension to your code.

import sys
import traceback

def func1():
    func2()

def func2():
    raise Exception('test error')

def main():

    try:
        func1()
    except:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        # Do your verification using exc_value and exc_traceback

        print "*** print_exception:"
        traceback.print_exception(exc_type, exc_value, exc_traceback,
                                  limit=3, file=sys.stdout)

if __name__ == '__main__':
    main()

This would print, similar to what you wanted.

*** print_exception:
Traceback (most recent call last):
  File "err_test.py", line 14, in main
    func1()
  File "err_test.py", line 5, in func1
    func2()
  File "err_test.py", line 8, in func2
    raise Exception('test error')
Exception: test error

Pile

No, I don't want to print it in the main(). I want to re-raise it with the original traceback and let caller of main() to handle it (e.g. ignore, print to the console, save into the db, etc). Jochen's solution worked. – Aalesund

This would be the best answer for Python3, if you change the print to something like raise exc_type.with_traceback(exc_value, exc_traceback) – Bridlewise

Answer 4

While @Jochen's answer works well in the simple case, it is not capable of handling more complex cases, where you are not directly catching and rethrowing, but are for some reason given the exception as an object and wish to re-throw in a completely new context (i.e. if you need to handle it in a different process).

In this case, I propose the following:

1. get the original exc_info
2. format the original error message, with stack trace
3. throw a new exception with that full error message (stack trace incl.) embedded

Before you do this, define a new exception type that you will rethrow later...

class ChildTaskException(Exception):
    pass

In the offending code...

import sys
import traceback

try:
    # do something dangerous
except:
    error_type, error, tb = sys.exc_info()
    error_lines = traceback.format_exception(error_type, error, tb)
    error_msg = ''.join(error_lines)
    # for example, if you are doing multiprocessing, you might want to send this to another process via a pipe
    connection.send(error_msg)

Rethrow...

# again, a multiprocessing example of receiving that message through a pipe
error_msg = pcon.recv()
raise ChildTaskException(error_msg)

Suggest

Answer 5

In Python 3:

import sys

class CustomError(Exception):
    pass

try:
    code_throwing_an_exception()
except Exception as e:
    _, value, traceback = sys.exc_info()
    raise CustomError("A new Exception was raised: %s" % value).with_traceback(traceback)

Snip

Answer 6

Your main function needs to look like this:

def main():
    try:
        func1()
    except Exception, err:
        # error processing
        raise

This is the standard way of handling (and re-raising) errors. Here is a codepad demonstration.

Titanism

I have a feeling that except Exception, err: can be bypassed with old-style raise "bad exception" way of raising exceptions – Aalesund

@Aalesund then use except object, err – Titanism

It's not any different from err = sys.exc_info()[1]. Anyway, the main point was to re-raise err outside of except block without loosing original traceback. Jochen's solution worked. – Aalesund

Answer 7

As Jochen mentioned, Python can't know Whether err is a new exception or a caught exception that is being re-raised. The from clause can help here. I have a test.py that has this code:

try:
    raise RuntimeError("Original error")
except RuntimeError as e:
    err = e

raise RuntimeError("Previous operation failed") from err

The result is reasonable:

$ python test.py
Traceback (most recent call last):
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 2, in <module>
    raise RuntimeError("Original error")
RuntimeError: Original error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 6, in <module>
    raise RuntimeError("Previous operation failed") from err
RuntimeError: Previous operation failed

See this for the support of from in Python2.

If the chained representation of the exception is not ideal in your specific application, I can think of two alternative solutions.

try:
    raise RuntimeError("Original error")
except RuntimeError as e:
    err = e

raise err from None

Result:

$ python test.py
Traceback (most recent call last):
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 6, in <module>
    raise err from None
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 2, in <module>
    raise RuntimeError("Original error")
RuntimeError: Original error

This is not the original traceback but it at least shows the correct source of the "original" exception, which is line 2.

And finally the second approach that doesn't chain the exceptions:

try:
    raise RuntimeError("Original error")
except RuntimeError as e:
    err = e

raise err.with_traceback(err.__traceback__)

which outputs:

$ python test.py
Traceback (most recent call last):
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 6, in <module>
    raise err.with_traceback(err.__traceback__)
  File "/home/valajeyhani/development/repositories/geotab/image_builder/test.py", line 2, in <module>
    raise RuntimeError("Original error")
RuntimeError: Original error

The result is very similar to the previous approach.

Global