traceback shows up until decorator

Question

This nice little Python decorator can configurably disabled decorated functions:

enabled = get_bool_from_config()

def run_if_enabled(fn):
    def wrapped(*args, **kwargs):
        try:
            return fn(*args, **kwargs) if enabled else None
        except Exception:
            log.exception('')
            return None
    return wrapped

alas, if an exception is raised within fn() the traceback shows only up to the wrapper:

Traceback (most recent call last):
  File "C:\my_proj\run.py", line 46, in wrapped
    return fn(*args, **kwargs) if enabled else None
  File "C:\my_proj\run.py", line 490, in a_decorated_function
    some_dict['some_value']
KeyError: 'some_value'

1. Why?
2. Can I workaround to see the full traceback?

Cave

I can't seem to reproduce this. I get a full traceback if I raise a ValueError within a function that I decorate with save_if_allowed ... What are you doing that allows the function to raise a SyntaxError when it is run? (as opposed to when defined). – Trilobate

Can you post a full example? The two parts you've posted don't make sense… The stack trace would definitely include everything up to and including the line that caused the exception… And SyntaxErrors are raised at parse time, not run time, so the decorator should (normally) never see a SyntaxError. – Password

@DavidWolever -- You could get a (runtime) SyntaxError from various forms of eval/exec, but ... – Trilobate

Are you by any chance working with frozen source? – Gibberish

@Trilobate This too does not generate the traceback posted here... – Hypostatize

@Gibberish The .py extension suggests otherwise... – Hypostatize

Guys, I just used 0=1 to produce an exception as an example. The focus is on the traceback which for me is cut off as posted above @Hypostatize @Gibberish @DavidWolever @Trilobate – Cave

@Jonathan You better change that, because 0=1 is a SyntaxError, and if you have a syntax error, it will be the only line in the traceback. There is no way to have the output you had with such an exception (and all others give the full traceback as far as I can tell). – Hypostatize

@Jonathan right, but how are you creating that SyntaxError? Those aren't normally raised at runtime — they are raised at parse time — and the traceback won't contain any stack information (because there is no stack at parse time). – Password

@Jonathan Ok, in that case, there's something you aren't telling us ;) See my answer for an example. Please post a complete demonstration of the problem. – Password

I'm pretty certain your problem is that you have a decorator that is trying to call the function it tries to decorate at the time of decorating the function, which is wrong. Some versions of your code (now lost to the void due to the way SO loses history during rapid edits) have demonstrated this behavior, though at the precise second that I'm writing this comment, I see working code. – Sleepless

Ok, neat question! See my answer for the explanation and solution. – Password

Answer 1

Ah! Ok, now this is an interesting question!

Here is is the same approximate function, but grabbing the exception directly from sys.exc_info():

import sys
import traceback

def save_if_allowed(fn):
    def wrapped(*args, **kwargs):
        try:
            return fn(*args, **kwargs) if enabled else None
        except Exception:
            print "The exception:"
            print "".join(traceback.format_exception(*sys.exc_info()))
            return None
    return wrapped

@save_if_allowed
def stuff():
    raise Exception("stuff")


def foo():
    stuff()

foo()

And it's true: no higher stack frames are included in the traceback that's printed:

$ python test.py
The exception:
Traceback (most recent call last):
  File "x.py", line 21, in wrapped
    return fn(*args, **kwargs) if enabled else None
  File "x.py", line 29, in stuff
    raise Exception("stuff")
Exception: stuff

Now, to narrow this down a bit, I suspect it's happening because the stack frame only includes stack information up until the most recent try/except block… So we should be able to recreate this without the decorator:

$ cat test.py
def inner():
    raise Exception("inner")

def outer():
    try:
        inner()
    except Exception:
        print "".join(traceback.format_exception(*sys.exc_info()))

def caller():
    outer()

caller()

$ python test.py
Traceback (most recent call last):
  File "x.py", line 42, in outer
    inner()
  File "x.py", line 38, in inner
    raise Exception("inner")
Exception: inner

Ah ha! Now, on reflection, this does make sense in a certain kind of way: at this point, the exception has only encountered two stack frames: that of inner() and that of outer() — the exception doesn't yet know from whence outer() was called.

So, to get the complete stack, you'll need to combine the current stack with the exception's stack:

$ cat test.py
def inner():
    raise Exception("inner")

def outer():
    try:
        inner()
    except Exception:
        exc_info = sys.exc_info()
        stack = traceback.extract_stack()
        tb = traceback.extract_tb(exc_info[2])
        full_tb = stack[:-1] + tb
        exc_line = traceback.format_exception_only(*exc_info[:2])
        print "Traceback (most recent call last):"
        print "".join(traceback.format_list(full_tb)),
        print "".join(exc_line)

def caller():
    outer()

caller()

$ python test.py
Traceback (most recent call last):
  File "test.py", line 56, in <module>
    caller()
  File "test.py", line 54, in caller
    outer()
  File "test.py", line 42, in outer
    inner()
  File "test.py", line 38, in inner
    raise Exception("inner")
Exception: inner

See also:

• http://docs.python.org/2/library/sys.html
• http://docs.python.org/2/library/traceback.html

Password

In my effort to simplify the question, I did omit the try\except clause which I now added to the question, now I think it's identical – Cave

@Jonathan That explains it all! Replace log.exception(...) with raise, and you're done. – Hypostatize

oh, but I can't I must absorb the exception at this level, it is part of the functionality of this wrapper. The rest of the system assumes functions decorated by this decorator never raise exceptions – Cave

Ah! Now, this is interesting :) Let me see… – Password

@Jonathan Check the traceback module and logging module. They have what you're looking for, but I'm not sure exactly which function now... – Hypostatize

Normally you can use import traceback, sys; traceback.print_exception(*sys.exc_info()) to print a traceback… But @Jonathan is right — the higher-level frames aren't being included. Very weird. – Password

Thanks for reproducing it! so... Why does this happen?! – Cave

Ok! See my edits… I'm just working on figuring out the right way to glue the various sys and traceback functions together to get the complete stack. – Password