Faking a traceback in Python

Question

I'm writing a test runner. I have an object that can catch and store exceptions, which will be formatted as a string later as part of the test failure report. I'm trying to unit-test the procedure that formats the exception.

In my test setup, I don't want to actually throw an exception for my object to catch, mainly because it means that the traceback won't be predictable. (If the file changes length, the line numbers in the traceback will change.)

How can I attach a fake traceback to an exception, so that I can make assertions about the way it's formatted? Is this even possible? I'm using Python 3.3.

Simplified example:

class ExceptionCatcher(object):
    def __init__(self, function_to_try):
        self.f = function_to_try
        self.exception = None
    def try_run(self):
        try:
            self.f()
        except Exception as e:
            self.exception = e

def format_exception_catcher(catcher):
    pass
    # No implementation yet - I'm doing TDD.
    # This'll probably use the 'traceback' module to stringify catcher.exception


class TestFormattingExceptions(unittest.TestCase):
    def test_formatting(self):
        catcher = ExceptionCatcher(None)
        catcher.exception = ValueError("Oh no")

        # do something to catcher.exception so that it has a traceback?

        output_str = format_exception_catcher(catcher)
        self.assertEquals(output_str,
"""Traceback (most recent call last):
  File "nonexistent_file.py", line 100, in nonexistent_function
    raise ValueError("Oh no")
ValueError: Oh no
""")

Arv

Answer 1

Reading the source of traceback.py pointed me in the right direction. Here's my hacky solution, which involves faking the frame and code objects which the traceback would normally hold references to.

import traceback

class FakeCode(object):
    def __init__(self, co_filename, co_name):
        self.co_filename = co_filename
        self.co_name = co_name


class FakeFrame(object):
    def __init__(self, f_code, f_globals):
        self.f_code = f_code
        self.f_globals = f_globals


class FakeTraceback(object):
    def __init__(self, frames, line_nums):
        if len(frames) != len(line_nums):
            raise ValueError("Ya messed up!")
        self._frames = frames
        self._line_nums = line_nums
        self.tb_frame = frames[0]
        self.tb_lineno = line_nums[0]

    @property
    def tb_next(self):
        if len(self._frames) > 1:
            return FakeTraceback(self._frames[1:], self._line_nums[1:])


class FakeException(Exception):
    def __init__(self, *args, **kwargs):
        self._tb = None
        super().__init__(*args, **kwargs)

    @property
    def __traceback__(self):
        return self._tb

    @__traceback__.setter
    def __traceback__(self, value):
        self._tb = value

    def with_traceback(self, value):
        self._tb = value
        return self


code1 = FakeCode("made_up_filename.py", "non_existent_function")
code2 = FakeCode("another_non_existent_file.py", "another_non_existent_method")
frame1 = FakeFrame(code1, {})
frame2 = FakeFrame(code2, {})
tb = FakeTraceback([frame1, frame2], [1,3])
exc = FakeException("yo").with_traceback(tb)

print(''.join(traceback.format_exception(FakeException, exc, tb)))
# Traceback (most recent call last):
#   File "made_up_filename.py", line 1, in non_existent_function
#   File "another_non_existent_file.py", line 3, in another_non_existent_method
# FakeException: yo

Thanks to @User for providing FakeException, which is necessary because real exceptions type-check the argument to with_traceback().

This version does have a few limitations:

• It doesn't print the lines of code for each stack frame, as a real traceback would, because format_exception goes off to look for the real file that the code came from (which doesn't exist in our case). If you want to make this work, you need to insert fake data into linecache's cache (because traceback uses linecache to get hold of the source code), per @User's answer below.

• You also can't actually raise exc and expect the fake traceback to survive.

• More generally, if you have client code that traverses tracebacks in a different manner than traceback does (such as much of the inspect module), these fakes probably won't work. You'd need to add whatever extra attributes the client code expects.

These limitations are fine for my purposes - I'm just using it as a test double for code that calls traceback - but if you want to do more involved traceback manipulation, it looks like you might have to go down to the C level.

Arv

I agree with the C/Level but can help you out with the lines of the file if you need that, too. linecache.updatecache(filename, fake_module_with_a___loader___attribute) is the right direction. – Sebiferous

@Sebiferous Awesome! I'll look into that. – Arv

Just looked into you for that. Comment if you have a new solution and share it! :) – Sebiferous

Answer 2

EDIT2:

That is the code of linecache.. I will comment on it.

def updatecache(filename, module_globals=None): # module_globals is a dict
        # ...
    if module_globals and '__loader__' in module_globals:
        name = module_globals.get('__name__')
        loader = module_globals['__loader__']
            # module_globals = dict(__name__ = 'somename', __loader__ = loader)
        get_source = getattr(loader, 'get_source', None) 
            # loader must have a 'get_source' function that returns the source

        if name and get_source:
            try:
                data = get_source(name)
            except (ImportError, IOError):
                pass
            else:
                if data is None:
                    # No luck, the PEP302 loader cannot find the source
                    # for this module.
                    return []
                cache[filename] = (
                    len(data), None,
                    [line+'\n' for line in data.splitlines()], fullname
                )
                return cache[filename][2]

That means before you testrun just do:

class Loader:
    def get_source(self):
        return 'source of the module'
import linecache
linecache.updatecache(filename, dict(__name__ = 'modulename without <> around', 
                                     __loader__ = Loader()))

and 'source of the module' is the source of the module you test.

EDIT1:

My solution so far:

class MyExeption(Exception):
    _traceback = None
    @property
    def __traceback__(self):
        return self._traceback
    @__traceback__.setter
    def __traceback__(self, value):
        self._traceback = value
    def with_traceback(self, tb_or_none):
        self.__traceback__ = tb_or_none
        return self

Now you can set the custom tracebacks of the exception:

e = MyExeption().with_traceback(1)

What you usually do if you reraise an exception:

raise e.with_traceback(fake_tb)

All exception prints walk through this function:

import traceback
traceback.print_exception(_type, _error, _traceback)

Hope it helps somehow.

Sebiferous

What sort of object do you pass to with_traceback()? – Arv

Answer 3

You should be able to simply raise whatever fake exception you want where you want it in your test runs. The python exception docs suggest you create a class and raise that as your exception. It's section 8.5 of the docs.

http://docs.python.org/2/tutorial/errors.html

Should be pretty straightforward once you've got the class created.

Resent

-1. This doesn't address my question. I don't want to raise the exception from the test code directly, because the traceback will be unpredictable wrt line numbers and so on. It's the traceback I want to fake, not the class of the exception. – Arv

If you're not okay with creating a traceback and calling it with an exception that you force I don't see how you'll be able to "fake" a traceback. If you're trying to grab that traceback as you say in your question using catcher.exception you're going to have to output an exception to do so. Even if you created a fake traceback to guess it's format what would that get you if you can't test it's implementation through your actual module? – Resent

Python lets you inspect its runtime stack. See sys._getframe() and inspect.stack(). Tracebacks are first-class objects which you can traverse manually if you like (you can get hold of a live traceback through exception.__traceback__ or sys.exc_info()). With that in mind, it's not too extreme to expect that you might be able to make your own tracebacks in pure Python without throwing a real exception. – Arv

The idea is to test the reporting code separately from the business logic. I expect to be able to change the behaviour of the ExceptionCatcher without breaking the formatter's tests and vice versa, so the tests should have no interdependency. What's more, as I detailed in my question, 'real' traceback objects are hard to test - if I throw an exception from a certain line in the test file, and then the test file changes length, I have to update the line number in the assertion (or somehow make the assertion line-number-agnostic, perhaps using a regular expression). – Arv