How to access "__" (double underscore) variables in methods added to a class

Question

Background

I wish to use a meta class in order to add helper methods based on the original class. If the method I wish to add uses self.__attributeName I get an AttributeError (because of name mangling) but for an existing identical method this isn't a problem.

Code example

Here is a simplified example

# Function to be added as a method of Test
def newfunction2(self):
    """Function identical to newfunction"""
    print self.mouse
    print self._dog
    print self.__cat

class MetaTest(type):
    """Metaclass to process the original class and
    add new methods based on the original class
    """
    def __new__(meta, name, base, dct):
        newclass = super(MetaTest, meta).__new__(
                meta, name, base, dct
                )

        # Condition for adding newfunction2
        if "newfunction" in dct:
            print "Found newfunction!"
            print "Add newfunction2!"
            setattr(newclass, "newfunction2", newfunction2)

        return newclass

# Class to be modified by MetaTest
class Test(object):
    __metaclass__ = MetaTest

    def __init__(self):
        self.__cat = "cat"
        self._dog = "dog"
        self.mouse = "mouse"

    def newfunction(self):
        """Function identical to newfunction2"""
        print self.mouse
        print self._dog
        print self.__cat

T = Test()
T.newfunction()
T.newfunction2() # AttributeError: 'Test' object has no attribute '__cat'

Question

Is there a way of adding newfunction2 that could use self.__cat?

(Without renaming self.__cat to self._cat.)

And maybe something more fundamental, why isn't self.__cat being treated in the same way for both cases since newfunction2 is now part of Test?

Pouncey

Related: What is the meaning of a single- and a double-underscore before an object name? – Peking

Possible duplicate of What is the meaning of a single- and a double-underscore before an object name? – Peking

Answer 1

Name mangling happens when the methods in a class are compiled. Attribute names like __foo are turned in to _ClassName__foo, where ClassName is the name of the class the method is defined in. Note that you can use name mangling for attributes of other objects!

In your code, the name mangling in newfunction2 doesn't work because when the function is compiled, it's not part of the class. Thus the lookups of __cat don't get turned into __Test_cat the way they did in Test.__init__. You could explicitly look up the mangled version of the attribute name if you want, but it sounds like you want newfunction2 to be generic, and able to be added to multiple classes. Unfortunately, that doesn't work with name mangling.

Indeed, preventing code not defined in your class from accessing your attributes is the whole reason to use name mangling. Usually it's only worth bothering with if you're writing a proxy or mixin type and you don't want your internal-use attributes to collide with the attributes of the class you're proxying or mixing in with (which you won't know in advance).

Mesopause

In this docs tutorial of Django docs.djangoproject.com/en/2.0/intro/tutorial02 . There is this code line ' Question.objects.get(pub_date__year=current_year)' and pub date year var has a double underscore, is this the same as you have mentioned?@Mesopause – Zola

@LordDraagon: No, that appears to be Django's own thing. Its spelling may be partly inspired by Python's native name mangling, but it's implemented separately, and has different meaning. I think the double underscores are sort of like dots in attribute notation (so pub_date__year is sort of like pub_date.year). I don't know enough about Django to give a more complete explanation. – Mesopause

Answer 2

To answer both of your questions:

1. You will need to change self.__cat when you need to call it from newfunction2 to self._Test__cat thanks to the name mangling rule.

---

2. Python documentation:

This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.

Let me brake it down for you, it's saying that it doesn't matter where your interpreter is reading when it encounters a name mangled name. The name will only be mangled if it occurs in the definition of a class, which in your case, it's not. Since it's not directly "under" a class definition. So when it reads self.__cat, it's keeping it at self.__cat, not going to textually replace it with self._Test__cat since it isn't defined inside theTest class.

Temporize

Answer 3

You can use <Test instance>._Test__cat to access the __cat attribute from the Test class. (where <Test instance> is replaced by self or any other instance of the Test class)

learn more in the Python doc

Zonation

I thought of this, but since I am adding newmethod2 as a method, I thought the behaviour should be the same. i.e. that I can access self.__cat. – Pouncey

Answer 4

class B:
    def __init__(self):
        self.__private = 0 
    def __private_method(self):
        '''A private method via inheritance'''
        return ('{!r}'.format(self))
    def internal_method(self):
        return ('{!s}'.format(self))

class C(B):
    def __init__(self):
        super().__init__()
        self.__private = 1
    def __private_method(self):
        return 'Am from class C'

c = C()
print(c.__dict__)
b = B()
print(b.__dict__)
print(b._B__private)
print(c._C__private_method())

Housekeeping

Could you please add some explanation for why this solves the original poster’s problem. – Tergiversate