Re-raise exception with a different type and message, preserving existing information

Question

I'm writing a module and want to have a unified exception hierarchy for the exceptions that it can raise (e.g. inheriting from a FooError abstract class for all the foo module's specific exceptions). This allows users of the module to catch those particular exceptions and handle them distinctly, if needed. But many of the exceptions raised from the module are raised because of some other exception; e.g. failing at some task because of an OSError on a file.

What I need is to “wrap” the exception caught such that it has a different type and message, so that information is available further up the propagation hierarchy by whatever catches the exception. But I don't want to lose the existing type, message, and stack trace; that's all useful information for someone trying to debug the problem. A top-level exception handler is no good, since I'm trying to decorate the exception before it makes its way further up the propagation stack, and the top-level handler is too late.

This is partly solved by deriving my module foo's specific exception types from the existing type (e.g. class FooPermissionError(OSError, FooError)), but that doesn't make it any easier to wrap the existing exception instance in a new type, nor modify the message.

Python's PEP 3134 “Exception Chaining and Embedded Tracebacks” discusses a change accepted in Python 3.0 for “chaining” exception objects, to indicate that a new exception was raised during the handling of an existing exception.

What I'm trying to do is related: I need it also working in earlier Python versions, and I need it not for chaining, but only for polymorphism. What is the right way to do this?

Pretender

Exceptions already are completely polymorphic -- they're all subclasses of Exception. What are you trying to do? "Different message" is fairly trivial with a top-level exception handler. Why are you changing the class? – Beleaguer

As explained in the question (now, thanks for your comment): I'm trying to decorate an exception I've caught, so that it can propagate further up with more information but not losing any. A top-level handler is too late. – Pretender

Please have a look at my CausedException class which can do what you want in Python 2.x. Also in Python 3 it can be of use in case you want to give more than one original exception as cause of your exception. Maybe it fits your needs. – Vaudois

bignose has the python-3 solution. @DevinJeanpierre has the python-2 solution (kind of). – Lucrece

For python-2 I do something similar to @DevinJeanpierre but I am just appending a new string message: except Exception as e --> raise type(e), type(e)(e.message + custom_message), sys.exc_info()[2] --> this solution is from another SO question. This is not pretty but functional. – Lucrece

Related (not dupe): How do I raise the same Exception with a custom message in Python? – Germanium

Answer 1

Python 3 introduced exception chaining (as described in PEP 3134). This allows, when raising an exception, to cite an existing exception as the “cause”:

try:
    frobnicate()
except KeyError as exc:
    raise ValueError("Bad grape") from exc

The caught exception (exc, a KeyError) thereby becomes part of (is the “cause of”) the new exception, a ValueError. The “cause” is available to whatever code catches the new exception.

By using this feature, the __cause__ attribute is set. The built-in exception handler also knows how to report the exception's “cause” and “context” along with the traceback.

---

In Python 2, it appears this use case has no good answer (as described by Ian Bicking and Ned Batchelder). Bummer.

Pretender

Doesn't Ian Bicking describe my solution? I regret that I gave such a godawful answer, but it's weird that this one got accepted. – Prober

@bignose: Thank you for mentioning this. I missed this change in Python3. Exception chaining will save a lot grief. – Spencer

@Pretender You got my point not only from being right, but for the use of "frobnicate" :) – Minotaur

Answer was how to re-raise the same exception, not new exception. It marked answer is not answer indeed. – Mcbride

Exception chaining is actually the default behavior now, in fact it is the opposite the issue, suppressing the first exception that requires work, see PEP 409 python.org/dev/peps/pep-0409 – Intertwine

How would you accomplish this in python 2? – Harts

It seems to work fine (python 2.7) try: return 2 / 0 except ZeroDivisionError as e: raise ValueError(e) – Griskin

Answer 2

You can use sys.exc_info() to get the traceback, and raise your new exception with said traceback (as the PEP mentions). If you want to preserve the old type and message, you can do so on the exception, but that's only useful if whatever catches your exception looks for it.

For example

import sys

def failure():
    try: 1/0
    except ZeroDivisionError, e:
        type, value, traceback = sys.exc_info()
        raise ValueError, ("You did something wrong!", type, value), traceback

Of course, this is really not that useful. If it was, we wouldn't need that PEP. I'd not recommend doing it.

Prober

Devin, you store a reference to the traceback there, shouldn't you be explicitly deleting that reference? – Cattan

I didn't store anything, I left traceback as a local variable that presumably falls out of scope. Yes, it is conceivable that it doesn't, but if you raise exceptions like that in the global scope rather than within functions, you've got bigger issues. If your complaint is only that it could be executed in a global scope, the proper solution is not to add irrelevant boilerplate that has to be explained and isn't relevant for 99% of uses, but to rewrite the solution so that no such thing is necessary while making it seem as if nothing is different-- as I have now done. – Prober

Arafangion may be referring to a warning in the Python documentation for sys.exc_info(), @Devin. It says, "Assigning the traceback return value to a local variable in a function that is handling an exception will cause a circular reference." However, a following note says that since Python 2.2, the cycle can be cleaned up, but it's more efficient to just avoid it. – Tuckerbag

More details on different ways to re-raise exceptions in Python from two enlightened pythonistas: Ian Bicking and Ned Batchelder – Lesialesion

Answer 3

You could create your own exception type that extends whichever exception you've caught.

class NewException(CaughtException):
    def __init__(self, caught):
        self.caught = caught

try:
    ...
except CaughtException as e:
    ...
    raise NewException(e)

But most of the time, I think it would be simpler to catch the exception, handle it, and either raise the original exception (and preserve the traceback) or raise NewException(). If I were calling your code, and I received one of your custom exceptions, I'd expect that your code has already handled whatever exception you had to catch. Thus I don't need to access it myself.

Edit: I found this analysis of ways to throw your own exception and keep the original exception. No pretty solutions.

Recover

The use case I've described isn't for handling the exception; it's specifically about not handling it, but adding some extra information (an additional class and a new message) so that it can be handled further up the call stack. – Pretender

Blog post (this analysis) url has changed. Now located at: ianbicking.org/blog/2007/09/re-raising-exceptions.html – Strop

Answer 4

I also found that many times i need some "wrapping" to errors raised.

This included both in a function scope and sometimes wrap only some lines inside a function.

Created a wrapper to be used a decorator and context manager:

---

Implementation

import inspect
from contextlib import contextmanager, ContextDecorator
import functools    

class wrap_exceptions(ContextDecorator):
    def __init__(self, wrapper_exc, *wrapped_exc):
        self.wrapper_exc = wrapper_exc
        self.wrapped_exc = wrapped_exc

    def __enter__(self):
        pass

    def __exit__(self, exc_type, exc_val, exc_tb):
        if not exc_type:
            return
        try:
            raise exc_val
        except self.wrapped_exc:
            raise self.wrapper_exc from exc_val

    def __gen_wrapper(self, f, *args, **kwargs):
        with self:
            for res in f(*args, **kwargs):
                yield res

    def __call__(self, f):
        @functools.wraps(f)
        def wrapper(*args, **kw):
            with self:
                if inspect.isgeneratorfunction(f):
                    return self.__gen_wrapper(f, *args, **kw)
                else:
                    return f(*args, **kw)
        return wrapper

Usage examples

decorator

@wrap_exceptions(MyError, IndexError)
def do():
   pass

when calling do method, don't worry about IndexError, just MyError

try:
   do()
except MyError as my_err:
   pass # handle error 

context manager

def do2():
   print('do2')
   with wrap_exceptions(MyError, IndexError):
       do()

inside do2, in the context manager, if IndexError is raised, it will be wrapped and raised MyError

Wristlet

Please explain what "wrapping" will do to the original exception. What is the purpose of your code, and what behaviour does it enable? – Finicky

@Finicky - added some examples, hope it helps – Wristlet

Answer 5

This is tangential, but when building consistent error messages for my own library, I found we were wrapping our own error messages as they percolated up. Was pleased to see that Python 3.11 now offers an add_note function to augment an existing error with additional information, which might also be useful.

For Python 3.11 add_note()

The add_note() method is added to BaseException. It can be used to enrich exceptions with context information that is not available at the time when the exception is raised. The added notes appear in the default traceback.

So we are now following this pattern:

try:
   some_risky_business()
except MyCustomException as ce:
  ce.add_note(f"Here is some more critical context!")
  raise se
except Exception as e:
  raise MyCustomException("Wow, didn't expect this error.") from e

Recency

FYI adding to the message, or replacing it entirely, is easy - in Python 3.x, exceptions have an args attribute - see https://mcmap.net/q/40405/-manually-raising-throwing-an-exception-in-python – Storz

Answer 6

To really "convert" the exception and avoid a context or cause as explained by @bignose's answer, you have to do some hoakey stuff (Python 3 below):

import sys

new_ex = None

try:
    something_that_raises_ValueError()
except ValueError:
    _, _, tb = sys.exc_info()
    new_ex = TypeError('This is really how I want to report this')

if new_ex is not None:
    raise new_ex.with_traceback(tb)

By passing in the traceback, you keep it pointing to where the problem occurred, not your raise statement.

This could probably be turned into a context to be made more reusable.

Note that if you only want to change the message, you can manipulate the args. I have these two functions for that:

def append_message(e_: Exception, msg: str):
    """
    Appends `msg` to the message text of the exception `e`.

    Parameters
    ----------
    e_: Exception
        An exception instance whose `args` will be modified to include `msg`.

    msg: str
        The string to append to the message.
    """
    if e_.args:
        # Exception was raised with arguments
        e_.args = (str(e_.args[0]) + msg,) + e_.args[1:]
    else:
        e_.args = (msg,)


def replace_message(e_: Exception, msg: str):
    """
    Replaces the exception message with `msg`.

    Parameters
    ----------
    e_: Exception
        An exception instance whose `args` will be modified to be `msg`.

    msg: str
        The string to replace the exception message with.
    """
    if e_.args:
        e_.args = (msg,) + e_.args[1:]
    else:
        e_.args = (msg,)

Storz

Answer 7

The most straighforward solution to your needs should be this:

try:
     upload(file_id)
except Exception as upload_error:
     error_msg = "Your upload failed! File: " + file_id
     raise RuntimeError(error_msg, upload_error)

In this way you can later print your message and the specific error throwed by the upload function

Durman

That catches and then throws away the exception object, so no, it doesn't meet the needs of the question. The question asks how to keep the existing exception and allow that same exception, with all the useful information it contains, to continue propagating up the stack. – Pretender