Inspect python class attributes

Question

I need a way to inspect a class so I can safely identify which attributes are user-defined class attributes. The problem is that functions like dir(), inspect.getmembers() and friends return all class attributes including the pre-defined ones like: __class__, __doc__, __dict__, __hash__. This is of course understandable, and one could argue that I could just make a list of named members to ignore, but unfortunately these pre-defined attributes are bound to change with different versions of Python therefore making my project volnerable to changed in the python project - and I don't like that.

example:

>>> class A:
...   a=10
...   b=20
...   def __init__(self):
...     self.c=30
>>> dir(A)
['__doc__', '__init__', '__module__', 'a', 'b']
>>> get_user_attributes(A)
['a','b']

In the example above I want a safe way to retrieve only the user-defined class attributes ['a','b'] not 'c' as it is an instance attribute. So my question is... Can anyone help me with the above fictive function get_user_attributes(cls)?

I have spent some time trying to solve the problem by parsing the class in AST level which would be very easy. But I can't find a way to convert already parsed objects to an AST node tree. I guess all AST info is discarded once a class has been compiled into bytecode.

Ovarian

You mention that you've been trying to do it in the AST. Does that mean that you want only attributes that are defined immediately on the class and not on it's superclasses as well? I recognize that you don't want the 'builtin' ones but I'm confused on this issue. – Gallous

Answer 1

Below is the hard way. Here's the easy way. Don't know why it didn't occur to me sooner.

import inspect

def get_user_attributes(cls):
    boring = dir(type('dummy', (object,), {}))
    return [item
            for item in inspect.getmembers(cls)
            if item[0] not in boring]

---

Here's a start

def get_user_attributes(cls):
    boring = dir(type('dummy', (object,), {}))
    attrs = {}
    bases = reversed(inspect.getmro(cls))   
    for base in bases:
        if hasattr(base, '__dict__'):
            attrs.update(base.__dict__)
        elif hasattr(base, '__slots__'):
            if hasattr(base, base.__slots__[0]): 
                # We're dealing with a non-string sequence or one char string
                for item in base.__slots__:
                    attrs[item] = getattr(base, item)
            else: 
                # We're dealing with a single identifier as a string
                attrs[base.__slots__] = getattr(base, base.__slots__)
    for key in boring:
        del attrs['key']  # we can be sure it will be present so no need to guard this
    return attrs

This should be fairly robust. Essentially, it works by getting the attributes that are on a default subclass of object to ignore. It then gets the mro of the class that's passed to it and traverses it in reverse order so that subclass keys can overwrite superclass keys. It returns a dictionary of key-value pairs. If you want a list of key, value tuples like in inspect.getmembers then just return either attrs.items() or list(attrs.items()) in Python 3.

If you don't actually want to traverse the mro and just want attributes defined directly on the subclass then it's easier:

def get_user_attributes(cls):
    boring = dir(type('dummy', (object,), {}))
    if hasattr(cls, '__dict__'):
        attrs = cls.__dict__.copy()
    elif hasattr(cls, '__slots__'):
        if hasattr(base, base.__slots__[0]): 
            # We're dealing with a non-string sequence or one char string
            for item in base.__slots__:
                attrs[item] = getattr(base, item)
            else: 
                # We're dealing with a single identifier as a string
                attrs[base.__slots__] = getattr(base, base.__slots__)
    for key in boring:
        del attrs['key']  # we can be sure it will be present so no need to guard this
    return attrs

Gallous

don't think so. I was just playing devil's advocate then as I was just thinking about __slots__ at the time (trying to figure out some stuff with PyPy) – Tabby

This one was what I needed :-) thanks aaronasterling: boring = dir(type('dummy', (object,), {})) – Ovarian

@Jakob Simon-Gaarde Thanks for the comment. It gave me one of those "I'm an idiot" flashes. See my update. – Gallous

I like the idea of dir: unlike __dict__, it can be overridden in __dir()__ to expose attributes generated through getattr. The one thing that bothers me is that dir is described in the docs as a "convenience for for use at an interactive prompt" rather than a "rigorous or consistently defined set of names", and "its detailed behavior may change between implementations". But I suppose not much we can do about it. – Lollop

boring could be a global constant, no? – Caryl

What about the builtin vars function? – Utu

Answer 2

Double underscores on both ends of 'special attributes' have been a part of python before 2.0. It would be very unlikely that they would change that any time in the near future.

class Foo(object):
  a = 1
  b = 2

def get_attrs(klass):
  return [k for k in klass.__dict__.keys()
            if not k.startswith('__')
            and not k.endswith('__')]

print get_attrs(Foo)

['a', 'b']

Supposal

what about user defined __add__, __mul__, __iter__, etc.? – Gallous

If they are user-defined, then I want those too. Is it possible to aquire the AST tree for a class that is already parsed and byte-compiled? – Ovarian

@jakob, nope, it's impossible to get AST for "alive" code, as source no longer retains in memory after being parsed, and there cases when Python runs out of bytecode even without source, so there can be no AST. – Wyon

Tiny bug: Foo.__dict__.keys() should be klass.__dict__.keys(). – Wary

Although not explicitly stated, I think that the OP wants to get only variables, not methods. You just need to add one additional check in your list comprehension: [... and not callable(klass.__dict__.get(k)] – Discard

Answer 3

Thanks aaronasterling, you gave me the expression i needed :-) My final class attribute inspector function looks like this:

def get_user_attributes(cls,exclude_methods=True):
  base_attrs = dir(type('dummy', (object,), {}))
  this_cls_attrs = dir(cls)
  res = []
  for attr in this_cls_attrs:
    if base_attrs.count(attr) or (callable(getattr(cls,attr)) and exclude_methods):
      continue
    res += [attr]
  return res

Either return class attribute variabels only (exclude_methods=True) or also retrieve the methods. My initial tests og the above function supports both old and new-style python classes.

/ Jakob

Ovarian

nice. One improvement that might be made is to switch the check for exclude_methods and callable(getattr(...)) so that callable only runs if the simple boolean check fails. – Gallous

Answer 4

If you use new style classes, could you simply subtract the attributes of the parent class?

class A(object):
    a = 10
    b = 20
    #...

def get_attrs(Foo):
    return [k for k in dir(Foo) if k not in dir(super(Foo))]

Edit: Not quite. __dict__,__module__ and __weakref__ appear when inheriting from object, but aren't there in object itself. You could special case these--I doubt they'd change very often.

Facelifting

Answer 5

Sorry for necro-bumping the thread. I'm surprised that there's still no simple function (or a library) to handle such common usage as of 2019.

I'd like to thank aaronasterling for the idea. Actually, set container provides a more straightforward way to express it:

class dummy:    pass

def abridged_set_of_user_attributes(obj):
    return set(dir(obj))-set(dir(dummy))

def abridged_list_of_user_attributes(obj):
    return list(abridged_set_of_user_attributes(obj))

The original solution using list comprehension is actually two level of loops because there are two in keyword compounded, despite having only one for keyword made it look like less work than it is.

Todo

Answer 6

This worked for me to include user defined attributes with __ that might be be found in cls.__dict__

import inspect

class A:
    __a = True
    
    def __init__(self, _a, b, c):
        self._a = _a
        self.b = b
        self.c = c 

    def test(self):
        return False

cls = A(1, 2, 3)

members = inspect.getmembers(cls, predicate=lambda x: not inspect.ismethod(x))
attrs = set(dict(members).keys()).intersection(set(cls.__dict__.keys()))
__attrs = {m[0] for m in members if m[0].startswith(f'_{cls.__class__.__name__}')}
attrs.update(__attrs)

This will correctly yield: {'_A__a', '_a', 'b', 'c'}

You can update to clean the cls.__class__.__name__ if you wish

Kerge

Quick and dirty, print(str(inspect.getmembers(cls, predicate=lambda x: not inspect.ismethod(x)))) is the ticket. Just replace cls with your object to see inside. – Huneycutt