class attribute changing value for no reason

Question

I have a problem with a program I am writing and i cannot for the life of me figure out what I am doing wrong. Ok so basically I am writing a program to extract data from an XML document and manipulate with class representations of the data.

Now, I have some added complexity in my program in that I am trying to be smart and use a descriptor (I am learning about them and thought I would try integrating them into my code)

Note: I shortened the problem to a self contained python script which can be run as is:

#!/usr/bin/env python
import inspect

class Numberise(object):
    def __init__(self, value=0, base=16):
        self.base = base
        self.value = value

    def __get__(self, obj, objtype):
        return self.value
    def __set__(self, obj, val):
        #self.value = self.extract_number(val)
        self.value = val
        print 'set value to:', self.value


class Register(object):
    width = Numberise(base=10)
    def __init__(self, width=16, name='unNamed'):
        super(Register, self).__init__()
        tuple_args = inspect.getargvalues(inspect.currentframe()) #shorthand
        for arg in tuple_args[0]:
            setattr(self, arg, tuple_args[3][arg])

if __name__ == "__main__":

    new_regs = [Register(width=i) for i in range(10)]
    for i,reg in enumerate(new_regs):
        reg.width = i


    for R in new_regs:
        print 'In extract(). Id:%s name:%s, width:%d'%(id(R), R.name, R.width)

When I run the script I get the following output:

C:\Users\gkuhn\Desktop>python test.py
set value to: 0
set value to: 1
set value to: 2
set value to: 3
set value to: 4
set value to: 5
set value to: 6
set value to: 7
set value to: 8
set value to: 9
set value to: 0
set value to: 1
set value to: 2
set value to: 3
set value to: 4
set value to: 5
set value to: 6
set value to: 7
set value to: 8
set value to: 9
In extract(). Id:48851280 name:unNamed, width:9
In extract(). Id:48852080 name:unNamed, width:9
In extract(). Id:48879472 name:unNamed, width:9
In extract(). Id:49285200 name:unNamed, width:9
In extract(). Id:49291504 name:unNamed, width:9
In extract(). Id:49291984 name:unNamed, width:9
In extract(). Id:49292016 name:unNamed, width:9
In extract(). Id:49292048 name:unNamed, width:9
In extract(). Id:49292080 name:unNamed, width:9
In extract(). Id:49292112 name:unNamed, width:9

What I'd like is for the width value to be individual to each register object that I have made. It looks like it is being shared. Should they not be individual?!

Below was my original question, you do not need to read it but I have left it here anyway. The actual issue is already discussed.

So in the snippet below, I am basically grabbing my newly created Register objects and adding them to an already made list.

    self.regs = []
    temps = []

    for register in self.ip_root:
        unrolled_regs = UnrollRegister(register)
        new_regs = unrolled_regs.convert()
        for R in new_regs:
            #print 'In extract(). Id:%s name:%s, width:%d'%(id(R), R.name, R.width)
            if 'extended' in R.name.lower():
                print 'In extract(). Id:%s name:%s, width:%d'%(id(R), R.name, R.width)
                temps.append(R)
                #print 'In extract(). Id:%s name:%s, width:%d'%(id(R), R.name, R.width)
                a = copy.deepcopy(R)
                #print type(R).__dict__
                #print temps

        #self.regs.extend(new_regs)
        self.regs += new_regs
        #self.regs.extend(unrolled_regs.convert())

    for n in temps:
        print '\tIn loop. Id:%s name:%s, width:%d'%(id(n), n.name, n.width)
        #print type(n).__dict__

Excuse the prints, I have been trying to figure this out!

The definition for the class Register is:

class Register(Base):
    width = Numberise(base=10)
    address = Numberise(base=10)
    def __init__(self, name='unNamed', width=16, description='No description provided', 
                 access='RW', address=0, visibility='Public', reset='async',
                 documentation=''):
        super(Register, self).__init__()
        tuple_args = inspect.getargvalues(inspect.currentframe()) #shorthand
        for arg in tuple_args[0]:
            setattr(self, arg, tuple_args[3][arg])

        self.bitfields = []

As mentioned, I am using a data descriptor for the width and address attributes. The definition for the Numberise descriptor is:

class Numberise(Base):
    def __init__(self, value=0, base=16):
        self.base = base
        self.value = self.extract_number(value) 

    def __get__(self, obj, objtype):
        return self.value
    def __set__(self, obj, val):
        self.value = self.extract_number(val)

    def extract_number(self,input):
        "try and get the value being represented"
        if type(input) == int: #its already a number
            return input
        else: #its a string
            RE = re.compile(r"\d?'([hHdDbB])(\w+)") #of the form 'h10 (verilog)
            result = RE.search(input)
            if result is not None:
                radix, string_num = result.groups()
                return int(string_num, {'h':16, 'd':10, 'b':2}[radix.lower()])
            else:
                return int(input, self.base)

Base does not include much and I've included it here for clarity:

class Base(object):
    def __init__(self):
        self._parent_spacer = ''
        self._spacer = '\t'

    @property
    def parent_spacer(self):
        return self._parent_spacer
    @parent_spacer.setter
    def parent_spacer(self, value):
        self._parent_spacer = value
    @property
    def spacer(self):
        return self.parent_spacer+'\t'

The idea behind this descriptor is to ensure that no matter what we initialise the width and address attributes to be, the saved values will always be integers as opposed to strings.

Now the all important output after running the code:

In extract(). Id:239825680 name:ASIC_ADC_RESULTS_EXTENDED_READ, width:64
In extract(). Id:239779088 name:ASIC_HART_EXTENDED_RECEIVE_BUFFER, width:64
        In loop. Id:239825680 name:ASIC_ADC_RESULTS_EXTENDED_READ, width:16
        In loop. Id:239779088 name:ASIC_HART_EXTENDED_RECEIVE_BUFFER, width:16

Can someone save my sanity and explain this behavior to me?!

Acceptance

Could you expand on what you mean by "this behaviour"? What were you expecting to see? – Adamite

Answer 1

Note: this answer is similar to the OP's answer, but with a few differences worth noting.

After reading the article linked from another relevant SO question, I've come to the following code:

#!/usr/bin/env python
import inspect

class Numberise(object):
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, owner):
        if self.name not in instance.__dict__:
            raise (AttributeError, self.name)
        return '%o'%(instance.__dict__[self.name])

    def __set__(self, instance, value):
        print ('setting value to: %d'%value)
        instance.__dict__[self.name] = value

class Register(object):
    width = Numberise("truewidth")
    def __init__(self, width=16, name='unNamed'):
        super(Register, self).__init__()
        tuple_args = inspect.getargvalues(inspect.currentframe()) #shorthand
        for arg in tuple_args[0]:
            setattr(self, arg, tuple_args[3][arg])

if __name__ == "__main__":

    new_regs = [Register(width=i) for i in range(10)]
    for i,reg in enumerate(new_regs):
        reg.width = i

    for R in new_regs:
        print ('In extract(). Id:%s name:%s, width:%s, truewidth:%d'%(id(R), R.name, R.width, R.truewidth))

This program produces the output which I think is what's desired:

setting value to: 0
setting value to: 1
setting value to: 2
setting value to: 3
setting value to: 4
setting value to: 5
setting value to: 6
setting value to: 7
setting value to: 8
setting value to: 9
setting value to: 0
setting value to: 1
setting value to: 2
setting value to: 3
setting value to: 4
setting value to: 5
setting value to: 6
setting value to: 7
setting value to: 8
setting value to: 9
In extract(). Id:35542384 name:unNamed, width:0, truewidth:0
In extract(). Id:35543152 name:unNamed, width:1, truewidth:1
In extract(). Id:35537776 name:unNamed, width:2, truewidth:2
In extract(). Id:36072560 name:unNamed, width:3, truewidth:3
In extract(). Id:36070384 name:unNamed, width:4, truewidth:4
In extract(). Id:36073040 name:unNamed, width:5, truewidth:5
In extract(). Id:36073072 name:unNamed, width:6, truewidth:6
In extract(). Id:36073104 name:unNamed, width:7, truewidth:7
In extract(). Id:36073136 name:unNamed, width:10, truewidth:8
In extract(). Id:36073168 name:unNamed, width:11, truewidth:9

Here's an explanation of what happens. In the line width = Numberise("truewidth") of Register class, we introduce the descriptor. It is one per class, not one per instance, so no value is stored in Numberise itself: we got to store the actual values in the instances. The descriptor as it is defined allows us to access the member variable self.truewidth of an instance of Register class. For the purpose of illustration, the __get__ method returns not the truewidth (which would be return instance.__dict__[self.name]), but its string representation as an octal number. Printing R.width is accessing it via descriptor. Printing R.truewidth is accessing it directly.

We could have called the member variable width, the same as the descriptor, and there would be no naming conflict: the descriptor is a part of the class namespace, and the member variable is a part of each instance's namespace. So, truewidth is used only for clarity, to better distinguish the two entities. In real code, perhaps naming it width is better, so that the actual data is hidden behind the descriptor, and you can't access it by accident.

Also, the program was made both Python2 and Python3 friendly, just by adding parentheses to the lines with raise and print.

Amsterdam

Great answer Gassa, I realized my mistake after reading my linked article again but when I came back to fix it you had beaten me to it. Thanks for taking the time and explaining it in more detail. – Acceptance

Answer 2

Ok after reading this article: http://martyalchin.com/2007/nov/24/python-descriptors-part-2-of-2/ I have seen the error of my ways. In the aforementioned article, the following snippet is provided which illustrates the correct way to utilize a descriptor for this scenario:

class SimpleDescriptor(object):
   def __init__(self, name):
        self.name = name

    def __get__(self, instance, owner):
        if self.name not in instance.__dict__:
            raise AttributeError, self.name
        return instance.__dict__[self.name]

    def __set__(self, instance, value):
        instance.__dict__[self.name] = value

So my descriptor should be this:

class Numberise(object):
    def __init__(self, value=0, base=16):
        self.base = base
        self.value = value

    def __get__(self, obj, objtype):
        return obj.value
    def __set__(self, obj, val):
        #self.value = self.extract_number(val)
        obj.value = val
        print 'set value to:', self.value

I made the mistake by using the following class as a reference:

class RevealAccess(object):
    """A data descriptor that sets and returns values
       normally and prints a message logging their access.
    """

    def __init__(self, initval=None, name='var'):
        self.val = initval
        self.name = name

    def __get__(self, obj, objtype):
        print 'Retrieving', self.name
        return self.val

    def __set__(self, obj, val):
        print 'Updating', self.name
        self.val = val

>>> class MyClass(object):
    x = RevealAccess(10, 'var "x"')
    y = 5

>>> m = MyClass()
>>> m.x
Retrieving var "x"
10
>>> m.x = 20
Updating var "x"
>>> m.x
Retrieving var "x"
20
>>> m.y
5

The above class was taken from the documentation: http://docs.python.org/2/howto/descriptor.html Whilst the example is not wrong and it does what it should, it does not apply in this case obviously which was my mistake.

Acceptance

My answer is essentially the same, since I drew inspiration from the same article. Still, I figured it is worth posting anyway, for two reasons. First, I wanted to understand myself what's going on. Second, your code still has self.value in Numberise which is no longer used but can still be confusing. – Amsterdam