"Inner exception" (with traceback) in Python?

Question

My background is in C# and I've just recently started programming in Python. When an exception is thrown I typically want to wrap it in another exception that adds more information, while still showing the full stack trace. It's quite easy in C#, but how do I do it in Python?

Eg. in C# I would do something like this:

try
{
  ProcessFile(filePath);
}
catch (Exception ex)
{
  throw new ApplicationException("Failed to process file " + filePath, ex);
}

In Python I can do something similar:

try:
  ProcessFile(filePath)
except Exception as e:
  raise Exception('Failed to process file ' + filePath, e)

...but this loses the traceback of the inner exception!

Edit: I'd like to see both exception messages and both stack traces and correlate the two. That is, I want to see in the output that exception X occurred here and then exception Y there - same as I would in C#. Is this possible in Python 2.6? Looks like the best I can do so far (based on Glenn Maynard's answer) is:

try:
  ProcessFile(filePath)
except Exception as e:
  raise Exception('Failed to process file' + filePath, e), None, sys.exc_info()[2]

This includes both the messages and both the tracebacks, but it doesn't show which exception occurred where in the traceback.

Scaler

The accepted answer is getting out of date, perhaps you should consider accepting another one. – Bathymetry

@AaronHall unfortunately OP hasn't been seen around since 2015. – Renitarenitent

what if I do not want the outer trace? The wrapper is just there to add more info, I don't want to litter exception output with trace of (perfectly working) wrapper code! – Natalianatalie

Answer 1

Python 2

It's simple; pass the traceback as the third argument to raise.

import sys
class MyException(Exception): pass

try:
    raise TypeError("test")
except TypeError, e:
    raise MyException(), None, sys.exc_info()[2]

Always do this when catching one exception and re-raising another.

Pageantry

Thanks. That preserves the traceback, but it loses the error message of the original exception. How do I see both messages and both tracebacks? – Scaler

raise MyException(str(e)), ..., etc. – Pageantry

Always only one trace is passed upwards. If the first catcher does some complicated things like calling other functions to re-raise the caught exception, the trace of these things is lost. To preserve more than one trace, you can have a look at my answer below. – Intort

Python 3 adds raise E() from tb and .with_traceback(...) – Nernst

@GlennMaynard it is a pretty old question, but the middle argument of the raise is the value to pass to the exception (in case the first argument is an exception class and not an instance). So if you want to swap exceptions, instead of doing raise MyException(str(e)), None, sys.exc_info()[2], it is better to use this: raise MyException, e.args, sys.exc_info()[2]. – Palaver

@Palaver TypeError: instance exception may not have a separate value; python2.7 – Solothurn

@JossefHarush, most probably your first argument is not an exception class but an instance. Anyway, it's been almost one year, and nowadays I wouldn't care at all whether to use raise MyException(*e.args), None, sys.exc_info()[2] or raise MyException, e.args, sys.exc_info()[2]. My current IDE suggests to use the former. – Palaver

A Python2 and 3 compliant way is possible using the future package: python-future.org/compatible_idioms.html#raising-exceptions E.g. from future.utils import raise_ and raise_(ValueError, None, sys.exc_info()[2]). – Baileybailie

Slightly off topic; is it less Pythonic to write except KeyError as e instead of except KeyError, e? – Publea

Is with_traceback(...) still legit? trying to find a proper example which raises a custom exception along with the chained exception log i.e. sys.exc_info()[2] – Moonstruck

Answer 2

Python 3

In python 3 you can do the following:

try:
    raise MyExceptionToBeWrapped("I have twisted my ankle")

except MyExceptionToBeWrapped as e:

    raise MyWrapperException("I'm not in a good shape") from e

This will produce something like this:

   Traceback (most recent call last):
   ...
   MyExceptionToBeWrapped: ("I have twisted my ankle")

The above exception was the direct cause of the following exception:

   Traceback (most recent call last):
   ...
   MyWrapperException: ("I'm not in a good shape")

Crowned

raise ... from ... is indeed the correct way to do this in Python 3. This needs more upvotes. – Tidewaiter

Nakedible I think it is because unfortunately most people still aren't using Python 3. – Entoblast

This seems to happen even with using 'from' in python 3 – Bordelon

Could be backported to Python 2. Hope it will one day. – Finable

Is there an alternative syntax to reach the same behavior under Python 3 but could also be included in a Python 2 + 3 compatible code base (without causing a SyntaxError under Python2)? – Pork

@Pork You can use the future package to achieve this: python-future.org/compatible_idioms.html#raising-exceptions E.g. from future.utils import raise_ and raise_(ValueError, None, sys.exc_info()[2]). – Baileybailie

@mwjohnson I undid your edit, the line The above exception was the direct cause of the following exception is part of the output in stderr, and so should be formatted as such. – Carrell

How do I get this "inner" exception in catch clause after I use the "raise... from..." syntax while throwing? – Blissful

Answer 3

Python 3 has the raise ... from clause to chain exceptions. Glenn's answer is great for Python 2.7, but it only uses the original exception's traceback and throws away the error message and other details. Here are some examples in Python 2.7 that add context information from the current scope into the original exception's error message, but keep other details intact.

Known Exception Type

try:
    sock_common = xmlrpclib.ServerProxy(rpc_url+'/common')
    self.user_id = sock_common.login(self.dbname, username, self.pwd)
except IOError:
    _, ex, traceback = sys.exc_info()
    message = "Connecting to '%s': %s." % (config['connection'],
                                           ex.strerror)
    raise IOError, (ex.errno, message), traceback

That flavour of raise statement takes the exception type as the first expression, the exception class constructor arguments in a tuple as the second expression, and the traceback as the third expression. If you're running earlier than Python 2.2, see the warnings on sys.exc_info().

Any Exception Type

Here's another example that's more general purpose if you don't know what kind of exceptions your code might have to catch. The downside is that it loses the exception type and just raises a RuntimeError. You have to import the traceback module.

except Exception:
    extype, ex, tb = sys.exc_info()
    formatted = traceback.format_exception_only(extype, ex)[-1]
    message = "Importing row %d, %s" % (rownum, formatted)
    raise RuntimeError, message, tb

Modify the Message

Here's another option if the exception type will let you add context to it. You can modify the exception's message and then reraise it.

import subprocess

try:
    final_args = ['lsx', '/home']
    s = subprocess.check_output(final_args)
except OSError as ex:
    ex.strerror += ' for command {}'.format(final_args)
    raise

That generates the following stack trace:

Traceback (most recent call last):
  File "/mnt/data/don/workspace/scratch/scratch.py", line 5, in <module>
    s = subprocess.check_output(final_args)
  File "/usr/lib/python2.7/subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory for command ['lsx', '/home']

You can see that it shows the line where check_output() was called, but the exception message now includes the command line.

Hunk

Where is the ex.strerror coming from? I can't find any relevant hit for that in the Python docs. Shouldn't it be str(ex)? – Sunfast

IOError is derived from EnvironmentError, @hheimbuerger, which provides the errorno and strerror attributes. – Hunk

How would I wrap an arbitrary Error, e.g. ValueError, into a RuntimeError by catching Exception? If I reproduce your answer for this case, the stacktrace is lost. – Presbyterial

I'm not sure what you're asking, @karl. Can you post a sample in a new question and then link to it from here? – Hunk

I edited my duplicate of the question of the OP at #23158266 with a clearification taking into account your answer directly. We should discuss there :) – Presbyterial

I see, @karl, you want to have the details of both stack traces, as you do with chained exceptions in Java. I don't know of any way to do that in Python 2, but it sounds like Alexei's answer covers how to do it in Python 3. – Hunk

Answer 4

In Python 3.x:

raise Exception('Failed to process file ' + filePath).with_traceback(e.__traceback__)

or simply

except Exception:
    raise MyException()

which will propagate MyException but print both exceptions if it will not be handled.

In Python 2.x:

raise Exception, 'Failed to process file ' + filePath, e

---

You can prevent printing both exceptions by killing the __context__ attribute. Here I write a context manager using that to catch and change your exception on the fly: (see http://docs.python.org/3.1/library/stdtypes.html for expanation of how they work)

try: # Wrap the whole program into the block that will kill __context__.

    class Catcher(Exception):
        '''This context manager reraises an exception under a different name.'''

        def __init__(self, name):
            super().__init__('Failed to process code in {!r}'.format(name))

        def __enter__(self):
            return self

        def __exit__(self, exc_type, exc_val, exc_tb):
            if exc_type is not None:
                self.__traceback__ = exc_tb
                raise self

    ...


    with Catcher('class definition'):
        class a:
            def spam(self):
                # not really pass, but you get the idea
                pass

            lut = [1,
                   3,
                   17,
                   [12,34],
                   5,
                   _spam]


        assert a().lut[-1] == a.spam

    ...


except Catcher as e:
    e.__context__ = None
    raise

Asshur

TypeError: raise: arg 3 must be a traceback or None – Pageantry

Sorry, I made a mistake, somehow I thought it also accepts exceptions and gets their traceback attribute automatically. As per docs.python.org/3.1/reference/…, this should be e.__traceback__ – Asshur

@ilyan.: Python 2 does not have e.__traceback__ attribute! – Clipping

Answer 5

I don't think you can do this in Python 2.x, but something similar to this functionality is part of Python 3. From PEP 3134:

In today's Python implementation, exceptions are composed of three parts: the type, the value, and the traceback. The 'sys' module, exposes the current exception in three parallel variables, exc_type, exc_value, and exc_traceback, the sys.exc_info() function returns a tuple of these three parts, and the 'raise' statement has a three-argument form accepting these three parts. Manipulating exceptions often requires passing these three things in parallel, which can be tedious and error-prone. Additionally, the 'except' statement can only provide access to the value, not the traceback. Adding the 'traceback' attribute to exception values makes all the exception information accessible from a single place.

Comparison to C#:

Exceptions in C# contain a read-only 'InnerException' property that may point to another exception. Its documentation [10] says that "When an exception X is thrown as a direct result of a previous exception Y, the InnerException property of X should contain a reference to Y." This property is not set by the VM automatically; rather, all exception constructors take an optional 'innerException' argument to set it explicitly. The 'cause' attribute fulfills the same purpose as InnerException, but this PEP proposes a new form of 'raise' rather than extending the constructors of all exceptions. C# also provides a GetBaseException method that jumps directly to the end of the InnerException chain; this PEP proposes no analog.

Note also that Java, Ruby and Perl 5 don't support this type of thing either. Quoting again:

As for other languages, Java and Ruby both discard the original exception when another exception occurs in a 'catch'/'rescue' or 'finally'/'ensure' clause. Perl 5 lacks built-in structured exception handling. For Perl 6, RFC number 88 [9] proposes an exception mechanism that implicitly retains chained exceptions in an array named @@.

Boulware

But, of course, in Perl5 you can just say "confess qq{OH NOES! $@}" and not lose the other exception's stack trace. Or you can implement your own type which retains the exception. – Hubing

Is this outdated or something? You can except Exception as err: then raise WhateverError('failed while processing ' + x) from err and the "innerException" is the from thing, right? – Fundamentalism

@Fundamentalism It is up to date information about Python 2 vs Python 3 – Sponger

Answer 6

For maximum compatibility between Python 2 and 3, you can use raise_from in the six library. https://six.readthedocs.io/#six.raise_from . Here is your example (slightly modified for clarity):

import six

try:
  ProcessFile(filePath)
except Exception as e:
  six.raise_from(IOError('Failed to process file ' + repr(filePath)), e)

Plug

Answer 7

You could use my CausedException class to chain exceptions in Python 2.x (and even in Python 3 it can be useful in case you want to give more than one caught exception as cause to a newly raised exception). Maybe it can help you.

Intort

Answer 8

Maybe you could grab the relevant information and pass it up? I'm thinking something like:

import traceback
import sys
import StringIO

class ApplicationError:
    def __init__(self, value, e):
        s = StringIO.StringIO()
        traceback.print_exc(file=s)
        self.value = (value, s.getvalue())

    def __str__(self):
        return repr(self.value)

try:
    try:
        a = 1/0
    except Exception, e:
        raise ApplicationError("Failed to process file", e)
except Exception, e:
    print e

Forbidden

Answer 9

Assuming:

• you need a solution, which works for Python 2 (for pure Python 3 see raise ... from solution)
• just want to enrich the error message, e.g. providing some additional context
• need the full stack trace

you can use a simple solution from the docs https://docs.python.org/3/tutorial/errors.html#raising-exceptions:

try:
    raise NameError('HiThere')
except NameError:
    print 'An exception flew by!' # print or log, provide details about context
    raise # reraise the original exception, keeping full stack trace

The output:

An exception flew by!
Traceback (most recent call last):
  File "<stdin>", line 2, in ?
NameError: HiThere

It looks like the key piece is the simplified 'raise' keyword that stands alone. That will re-raise the Exception in the except block.

Dun

This is the Python 2 & 3 compatible solution! Thanks! – Abide

I think the idea was to raise a different type of exception. – Entoblast

This is not a chain of nested exceptions, just reraising one exception – Presbyterial

This is the best python 2 solution, if you just need to enrich the exception message and have the full stack trace! – Olav

What is difference between using raise and raise from – Kaleena

The output goes to stdout. How to log this trace using logging? – Suburbia