Wrapping all possible method calls of a class in a try/except block

Question

I'm trying to wrap all methods of an existing Class (not of my creation) into a try/except suite. It could be any Class, but I'll use the pandas.DataFrame class here as a practical example.

So if the invoked method succeeds, we simply move on. But if it should generate an exception, it is appended to a list for later inspection/discovery (although the below example just issues a print statement for simplicity).

(Note that the kinds of data-related exceptions that can occur when a method on the instance is invoked, isn't yet known; and that's the reason for this exercise: discovery).

This post was quite helpful (particularly @martineau Python-3 answer), but I'm having trouble adapting it. Below, I expected the second call to the (wrapped) info() method to emit print output but, sadly, it doesn't.

#!/usr/bin/env python3

import functools, types, pandas

def method_wrapper(method):
    @functools.wraps(method)
    def wrapper(*args, **kwargs): #Note: args[0] points to 'self'.
        try:
            print('Calling: {}.{}()... '.format(args[0].__class__.__name__,
                                                method.__name__))
            return method(*args, **kwargs)
        except Exception:
            print('Exception: %r' % sys.exc_info()) # Something trivial.
            #<Actual code would append that exception info to a list>.
    return wrapper


class MetaClass(type):
    def __new__(mcs, class_name, base_classes, classDict):
        newClassDict = {}
        for attributeName, attribute in classDict.items():
            if type(attribute) == types.FunctionType: # Replace it with a
                attribute = method_wrapper(attribute) # decorated version.
            newClassDict[attributeName] = attribute
        return type.__new__(mcs, class_name, base_classes, newClassDict)

class WrappedDataFrame2(MetaClass('WrappedDataFrame',
                                  (pandas.DataFrame, object,), {}),
                                  metaclass=type):
    pass

print('Unwrapped pandas.DataFrame().info():')
pandas.DataFrame().info()

print('\n\nWrapped pandas.DataFrame().info():')
WrappedDataFrame2().info()
print()

This outputs:

Unwrapped pandas.DataFrame().info():
<class 'pandas.core.frame.DataFrame'>
Index: 0 entries
Empty DataFrame

Wrapped pandas.DataFrame().info():   <-- Missing print statement after this line.
<class '__main__.WrappedDataFrame2'>
Index: 0 entries
Empty WrappedDataFrame2

In summary,...

>>> unwrapped_object.someMethod(...)
# Should be mirrored by ...

>>> wrapping_object.someMethod(...)
# Including signature, docstring, etc. (i.e. all attributes); except that it
# executes inside a try/except suite (so I can catch exceptions generically).

Snorkel

P.S. If I delay in my response to comments or answer, it may be because I'm trying the suggestion out or trying to understand it first. =:) – Snorkel

Answer 1

long time no see. ;-) In fact it's been such a long time you may no longer care, but in case you (or others) do...

Here's something I think will do what you want. I've never answered your question before now because I don't have pandas installed on my system. However, today I decided to see if there was a workaround for not having it and created a trivial dummy module to mock it (only as far as I needed). Here's the only thing in it:

mockpandas.py:

""" Fake pandas module. """

class DataFrame:
    def info(self):
        print('pandas.DataFrame.info() called')
        raise RuntimeError('Exception raised')

Below is code that seems to do what you need by implementing @Blckknght's suggestion of iterating through the MRO—but ignores the limitations noted in his answer that could arise from doing it that way). It ain't pretty, but as I said, it seems to work with at least the mocked pandas library I created.

import functools
import mockpandas as pandas  # mock the library
import sys
import traceback
import types

def method_wrapper(method):
    @functools.wraps(method)
    def wrapper(*args, **kwargs): # Note: args[0] points to 'self'.
        try:
            print('Calling: {}.{}()... '.format(args[0].__class__.__name__,
                                                method.__name__))
            return method(*args, **kwargs)
        except Exception:
            print('An exception occurred in the wrapped method {}.{}()'.format(
                    args[0].__class__.__name__, method.__name__))
            traceback.print_exc(file=sys.stdout)
            # (Actual code would append that exception info to a list)

    return wrapper

class MetaClass(type):
    def __new__(meta, class_name, base_classes, classDict):
        """ See if any of the base classes were created by with_metaclass() function. """
        marker = None
        for base in base_classes:
            if hasattr(base, '_marker'):
                marker = getattr(base, '_marker')  # remember class name of temp base class
                break  # quit looking

        if class_name == marker:  # temporary base class being created by with_metaclass()?
            return  type.__new__(meta, class_name, base_classes, classDict)

        # Temporarily create an unmodified version of class so it's MRO can be used below.
        TempClass = type.__new__(meta, 'TempClass', base_classes, classDict)

        newClassDict = {}
        for cls in TempClass.mro():
            for attributeName, attribute in cls.__dict__.items():
                if isinstance(attribute, types.FunctionType):
                    # Convert it to a decorated version.
                    attribute = method_wrapper(attribute)
                    newClassDict[attributeName] = attribute

        return type.__new__(meta, class_name, base_classes, newClassDict)

def with_metaclass(meta, classname, bases):
    """ Create a class with the supplied bases and metaclass, that has been tagged with a
        special '_marker' attribute.
    """
    return type.__new__(meta, classname, bases, {'_marker': classname})

class WrappedDataFrame2(
        with_metaclass(MetaClass, 'WrappedDataFrame', (pandas.DataFrame, object))):
    pass

print('Unwrapped pandas.DataFrame().info():')
try:
    pandas.DataFrame().info()
except RuntimeError:
    print('  RuntimeError exception was raised as expected')

print('\n\nWrapped pandas.DataFrame().info():')
WrappedDataFrame2().info()

Output:

Unwrapped pandas.DataFrame().info():
pandas.DataFrame.info() called
  RuntimeError exception was raised as expected


Wrapped pandas.DataFrame().info():
Calling: WrappedDataFrame2.info()...
pandas.DataFrame.info() called
An exception occurred in the wrapped method WrappedDataFrame2.info()
Traceback (most recent call last):
  File "test.py", line 16, in wrapper
    return method(*args, **kwargs)
  File "mockpandas.py", line 9, in info
    raise RuntimeError('Exception raised')
RuntimeError: Exception raised

As the above illustrates, the method_wrapper() decoratored version is being used by methods of the wrapped class.

Adequate

Answer 2

Your metaclass only applies your decorator to the methods defined in classes that are instances of it. It doesn't decorate inherited methods, since they're not in the classDict.

I'm not sure there's a good way to make it work. You could try iterating through the MRO and wrapping all the inherited methods as well as your own, but I suspect you'd get into trouble if there were multiple levels of inheritance after you start using MetaClass (as each level will decorate the already decorated methods of the previous class).

Anthraquinone

is monkey patching directly pd.DataFrame one of the solutions? – Interlocutress

Monkey patching could be an option. I wrote the author of Pandas to get his take, too (outside of StackOverflow). Maybe he has some thoughts, and I'll happily share that when I get in touch with him. Trying not to go too far off the beaten path, yet. =:) – Snorkel

Thanks for the feedback @Anthraquinone . Will pondering your answer more. – Snorkel

Hi @Blckknght. Thank you for the interesting MRO approach answer. Regarding the valid concern you mentioned ('multiple levels of inheritance), do you mean as a result of possibly sub-classing "MetaClass" and/or 'WrappedDataFrame2' in the future? Or some other side-effect? (Btw, 'MetaClass' here is for this one-time/exclusive private use). Thanks. – Snorkel