How to fake/proxy a class in Python

Question

I wrote some wrapper which has another object as an attribute. This wrapper proxies (forwards) all attribute requests with __getattr__ and __setattr__ to the object stored as the attribute. What else do I need to provide for my proxy so that the wrapper looks like the wrapped class under usual circumstances?

I suppose I need to fix things like inheritance, maybe __repr__, ... What else do I need to take care of and how do I fix inheritance so that instanceof() works?

EDIT: My attempt to make a function proxy, however as I don't understand the recipe fully, it fails :(

setattr_=object.__setattr__
getattr_=object.__getattribute__

class Proxy(object):
    __slots__=["_func", "_params", "_kwargs", "_obj", "_loaded", "__weakref__"]
    def __init__(self, func, *params, **kwargs):
        setattr_(self, "_func", func)
        setattr_(self, "_params", params)
        setattr_(self, "_kwargs", kwargs)

        setattr_(self, "_obj", None)
        setattr_(self, "_loaded", False)

    def _get_obj(self):
        if getattr_(self, "_loaded")==False:
            print("Loading")
            setattr_(self, "_obj", getattr_(self, "_func")(*getattr_(self, "_params"), **getattr_(self, "_kwargs")))
            setattr_(self, "_loaded", True)

        return getattr_(self, "_obj")
    #
    # proxying (special cases)
    #
    def __getattribute__(self, name):
        return getattr(getattr_(self, "_get_obj")(), name)
    def __delattr__(self, name):
        delattr(getattr_(self, "_get_obj")(), name)
    def __setattr__(self, name, value):
        setattr(getattr_(self, "_get_obj")(), name, value)

    def __nonzero__(self):
        return bool(getattr_(self, "_get_obj")())
    def __str__(self):
        return str(getattr_(self, "_get_obj")())
    def __repr__(self):
        return repr(getattr_(self, "_get_obj")())

    #
    # factories
    #
    _special_names=[
        '__abs__', '__add__', '__and__', '__call__', '__cmp__', '__coerce__',
        '__contains__', '__delitem__', '__delslice__', '__div__', '__divmod__',
        '__eq__', '__float__', '__floordiv__', '__ge__', '__getitem__',
        '__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__iand__',
        '__idiv__', '__idivmod__', '__ifloordiv__', '__ilshift__', '__imod__',
        '__imul__', '__int__', '__invert__', '__ior__', '__ipow__', '__irshift__',
        '__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__',
        '__long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__',
        '__neg__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__',
        '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__',
        '__repr__', '__reversed__', '__rfloorfiv__', '__rlshift__', '__rmod__',
        '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__',
        '__rtruediv__', '__rxor__', '__setitem__', '__setslice__', '__sub__',
        '__truediv__', '__xor__', 'next',
    ]

    @classmethod
    def _create_class_proxy(cls, theclass):
        """creates a proxy for the given class"""

        def make_method(name):
            def method(self, *args, **kw):
                return getattr(getattr_(self, "_get_obj")(), name)(*args, **kw)
            return method

        namespace={}
        for name in cls._special_names:
            if hasattr(theclass, name):
                namespace[name]=make_method(name)
        return type("%s(%s)"%(cls.__name__, theclass.__name__), (cls,), namespace)

    def __new__(cls, obj, *args, **kwargs):
        """
        creates an proxy instance referencing `obj`. (obj, *args, **kwargs) are
        passed to this class' __init__, so deriving classes can define an 
        __init__ method of their own.
        note: _class_proxy_cache is unique per deriving class (each deriving
        class must hold its own cache)
        """
        try:
            cache=cls.__dict__["_class_proxy_cache"]
        except KeyError:
            cls._class_proxy_cache=cache={}
        try:
            theclass=cache[obj.__class__]
        except KeyError:
            cache[obj.__class__]=theclass=cls._create_class_proxy(obj.__class__)
        ins=object.__new__(theclass)
        theclass.__init__(ins, obj, *args, **kwargs)
        return ins

if __name__=='__main__':
    def t(x, y):
        print("Running t")
        return x+y

    a=Proxy(t, "a", "b")
    print("Go")
    print(a+"c")

Polyhymnia

Out of interest, why don't you just inherit? – Threephase

Because this wrapper is supposed to wrap all sorts of objects. It is a general proxy. – Polyhymnia

You've changed the recipe a bit. It sort of looks like what you really want is functools.partial, but maybe not; what is the problem you are really trying to solve? – Servais

I want to decorate a function so that it is executed lazily. It is for some kind of configuration file where I don't want to execute all value elements (function calls) from a dict. It can be in all kinds of nesting levels so that I don't want a specialized mechanism. Meanwhile I figured the recipe works if I remove the if hasattr(cls, name) line. However I'm not sure what I'm doing :) – Polyhymnia

Answer 1

This problem is reasonably well addressed by this recipe:

Object Proxying (Python recipe)

The general idea you have to follow is that most methods on classes are accessed through some combination of __getattr__ and __getattribute__ either on the class itself, or on its metaclass, but this does not apply to python special methods, the ones that begin and end with double underscore, for those to be found, they must be actual methods on the actual class, no attribute proxying is possible.

Which of those methods you must provide will obviously depend on which of those methods the proxied class itself offers. The special methods you need to provide for isinstance() are the __instancecheck__ and __subclasscheck__ methods. for repr() to work, you must also define an appropriate __repr__() on the proxy class itself.

Servais

That seems very interesting, however I find it very hard to understand all details :( I tried making it a proxy for function calls - see edit. However, I suppose I need to do the special names generation somehow after the object is initialized. Can help me understand what I need to change? – Polyhymnia

Answer 2

Generally, you can use the wrapt library (pypi), which does the heavy lifting for you:

The wrapt module focuses very much on correctness. It therefore goes way beyond existing mechanisms such as functools.wraps() to ensure that decorators preserve introspectability, signatures, type checking abilities etc. [...]

To ensure that the overhead is as minimal as possible, a C extension module is used for performance critical components

It supports creating custom wrapper classes. In order to add your own attributes, you need to declare them in such a way that wrapt doesn't try to pass them on to the wrapped instance. You can:

• Prefix the attribute with _self_ and add properties for access
• Declare the attribute at the class level, in addition to in __init__

• Use slots, if appropriate for your class (not mentioned in the docs), like this:

  class ExtendedMesh(ObjectProxy):
      __slots__ = ('foo')
  
      def __init__(self, subject):
          super().__init__(subject)
          self.foo = "bar"
  

It also supports function wrappers, which might suit your purpose.

Margiemargin

Answer 3

in case someone else is looking for a more complete proxy implementation

Although there are several python proxy solutions similar to those OP is looking for, I could not find a solution that will also proxy classes and arbitrary class objects, as well as automatically proxy functions return values and arguments. Which is what I needed.

I've got some code written for that purpose as part of a full proxying/logging python execution and I may make it into a separate library in the future. If anyone's interested you can find the core of the code in a pull request. Drop me a line if you'd like this as a standalone library.

My code will automatically return wrapper/proxy objects for any proxied object's attributes, functions and classes. The purpose is to log and replay some of the code, so i've got the equivalent "replay" code and some logic to store all proxied objects to a json file.

Capsulize

Answer 4

After some trying, I think I have arrived at a solution which does not need an external library or manually listing / implementing all kinds of special functions:

class ProxyMeta(type):
    def __new__(cls, wrapped_obj):
        mro = type(wrapped_obj).__mro__
        attrs = dict()
        for typ in reversed(mro):
            attrs.update(vars(typ))
        try:
            attrs.update(vars(wrapped_obj))
        except TypeError:
            pass
        dct = dict()
        for name, value in attrs.items():
            if name in ["__new__", "__init__"]:
                continue
            if callable(value):
                def wrap_func(self, *args, _orig_func=value, **kwargs):
                    return _orig_func(wrapped_obj, *args, **kwargs)
                dct[name] = wrap_func
            else:
                dct[name] = value
        return type(f"Proxy[{type(wrapped_obj).__name__}]", (), dct)

def Proxy(obj):
    return ProxyMeta(obj)()

p = Proxy([1, 2, 3])
print(type(p).__name__, str(p), len(p), f"{p}")
p = Proxy(True)
print(type(p).__name__, str(p),  f"{p}", p == True)
p = Proxy("asdf")
print(type(p).__name__, str(p), len(p), p.upper())
p = Proxy(42)
print(type(p).__name__, str(p), p == 42, p-1 == 41)

class A(list):
    def __init__(self):
        self.a = 2
        super().__init__([1,2,4])

p = Proxy(A())
print(type(p).__name__, vars(p), p)

Output:

Proxy[list] [1, 2, 3] 3 [1, 2, 3]
Proxy[bool] True True True
Proxy[str] asdf 4 ASDF
Proxy[int] 42 True True
Proxy[A] {'a': 2} [1, 2, 4]

I certainly wouldn't claim to be an expert on the Python data model, so this might fail in more complicated situations.

Overlarge