Python function attributes - uses and abuses [closed]

Question

Not many are aware of this feature, but Python's functions (and methods) can have attributes. Behold:

>>> def foo(x):
...     pass
...     
>>> foo.score = 10
>>> dir(foo)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name', 'score']
>>> foo.score
10
>>> foo.score += 1
>>> foo.score
11

What are the possible uses and abuses of this feature in Python ? One good use I'm aware of is PLY's usage of the docstring to associate a syntax rule with a method. But what about custom attributes ? Are there good reasons to use them ?

Borras

Check out PEP 232. – Zumstein

Is this very surprising? In general, Python objects support ad-hoc attributes. Of course, some do not, particularly those with builtin type. To me, those those that do not support this seem to be the exceptions, not the rule. – Borlase

Here are PEP 232's additional uses for function attributes – Sacring

One Application in Django: Customize the admin change list – Kenelm

@GrijeshChauhan I came to this question after seeing these docs! – Phthalocyanine

Pity that this is closed, I wanted to add that you can attach any custom exceptions that the function might raise, to provide easy access when catching it in the calling code. I'd provide an illustrative example, but that's best done in an answer. – Murr

@Borlase Yes it is surprising. In most languages a method is not an object. – Watts

Pity that this is closed. CherryPy is a nice example of how this is used. – Watts

Here's an example of use in the django source code – Audiometer

The Django docs were reorganized. The new link for @GrijeshChauhan's comment is Customize the admin change list – Perspiration

After 20+ years of python programming I've just used this feature for the first time. I wrote a function to be used as an argparse argument type (i.e. to parse string to some other type), and as it's one I'm anticipating reusing, I gave it a help attribute, containing a string which is a sensible default for an add_argument()'s help parameter in cases where this type is used. Very niche, but definitely handy. – Ashe

Answer 1

I typically use function attributes as storage for annotations. Suppose I want to write, in the style of C# (indicating that a certain method should be part of the web service interface)

class Foo(WebService):
    @webmethod
    def bar(self, arg1, arg2):
         ...

then I can define

def webmethod(func):
    func.is_webmethod = True
    return func

Then, when a webservice call arrives, I look up the method, check whether the underlying function has the is_webmethod attribute (the actual value is irrelevant), and refuse the service if the method is absent or not meant to be called over the web.

Repentant

Do you think there are down-sides to this? e.g. What if two libraries try to write the same ad-hoc attribute? – Borlase

I was thinking of doing exactly this. Then I stopped myself. "Is this a bad idea?" I wondered. Then, I wandered over to SO. After some bumbling around, I found this question/answer. Still not sure if this is a good idea. – Borlase

This is definitely the most legit use of function attributes of all the answers (as of Nov, 2012). Most (if not all) the other answers use function attributes as a replacement for global variables; however, they do NOT get rid of global state, which is exactly the problem with global variables. This is different, because once the value is set, it does not change; it is constant. A nice consequence of this is that you don't run into synchronization problems, which are inherent to global variables. Yes, you can provide your own synchronization, but that's the point: it's not safe automatically. – Borlase

Indeed, I say, as long as attribute doesn't change behaviour of the function in question, it's good. Compare to .__doc__ – Spratt

This approach can also be used to attach output description to the decorated function, which is missing in python 2.*. – Arredondo

How do I intervene during the method lookup to verify whether the being-called method is a webservice or not? in other words, where do I put the logic that says if webservice: # allow else: #don't – Apomixis

I have tried to do exactly this, but the @property.setter statement somehow removes the added attribute and it cannot be seen anymore. I posted my question here – Secrete

Answer 2

I've used them as static variables for a function. For example, given the following C code:

int fn(int i)
{
    static f = 1;
    f += i;
    return f;
}

I can implement the function similarly in Python:

def fn(i):
    fn.f += i
    return fn.f
fn.f = 1

This would definitely fall into the "abuses" end of the spectrum.

Leptorrhine

Interesting. Are there other ways to implement static variables in python? – Borras

You can use a global, too, but then the value is available...well, globally. – Leptorrhine

That's a pretty poor reason to downvote this answer, which is demonstrating an analogy between C and Python, not advocating the best possible way to write this particular function. – Bigamous

@Leptorrhine attributes on global functions are also global! – Borlase

@RobertRossney But if generators are the way to go, then this is a poor use of function attributes. If so, then this is an abuse. Not sure whether to upvote abuses though, since the question asks for those too :P – Borlase

@Leptorrhine interesting concept. While it's not really an analogue to C static, it can be quite useful, especially if fn is redefined and you expect to get new static state. Compare to using "global" dict(). – Spratt

Definitely seems like an abuse, per PEP 232, thanks @user140352, and hop's comment and this SO answer – Sacring

@Sacring I don't see how it's an abuse per PEP 232. PEP 232 provides some use cases for the mechanism, but doesn't seem to recommend that the use is limited to those use cases. – Jillane

@EvgeniSergeev, yea, you may be right. PEP 232 disproves the null hypothesis but doesn't prove the hypothesis. But it also initially seemed like a good abuse example, IMO. But now that I research and ponder, I can't think of a better way to implement static, so I can see why you see it as a good usage. – Sacring

adding caching functionality to allow persistent variables without dealing with classes, databases, or memcache – Bradshaw

I am not sure about the thought process of adding an attribute to maintain what C program does using static. Are we not getting into state issue by adding an attribute that store a value? Is there a standard reason, why language designers allowed adding attributes to function type objects in python? – Grogan

Answer 3

You can do objects the JavaScript way... It makes no sense but it works ;)

>>> def FakeObject():
...   def test():
...     print "foo"
...   FakeObject.test = test
...   return FakeObject
>>> x = FakeObject()
>>> x.test()
foo

Akim

+1 A fine example of an abuse of this feature, which is one of the things the question asked for. – Labret

How is this different from mipadi's answer? Seems to be the same thing, except instead of an int, the attribute value is a function. – Borlase

is def test() really necessary? – Helen

Answer 4

I use them sparingly, but they can be pretty convenient:

def log(msg):
   log.logfile.write(msg)

Now I can use log throughout my module, and redirect output simply by setting log.logfile. There are lots and lots of other ways to accomplish that, but this one's lightweight and dirt simple. And while it smelled funny the first time I did it, I've come to believe that it smells better than having a global logfile variable.

Bigamous

re smell: This doesn't get rid of global logfile though. It just squirrels it away in another global, the log function. – Borlase

@allyourcode: But it can help avoid name clashes if you have to have a bunch of global logfiles for different functions in the same module. – Behest

Answer 5

Function attributes can be used to write light-weight closures that wrap code and associated data together:

#!/usr/bin/env python

SW_DELTA = 0
SW_MARK  = 1
SW_BASE  = 2

def stopwatch():
   import time

   def _sw( action = SW_DELTA ):

      if action == SW_DELTA:
         return time.time() - _sw._time

      elif action == SW_MARK:
         _sw._time = time.time()
         return _sw._time

      elif action == SW_BASE:
         return _sw._time

      else:
         raise NotImplementedError

   _sw._time = time.time() # time of creation

   return _sw

# test code
sw=stopwatch()
sw2=stopwatch()
import os
os.system("sleep 1")
print sw() # defaults to "SW_DELTA"
sw( SW_MARK )
os.system("sleep 2")
print sw()
print sw2()

1.00934004784

2.00644397736

3.01593494415

Aphelion

Why push functions when we have classes handy? And let's not forget classes can emulate a function. – Ufo

also time.sleep(1) is better than os.system('sleep 1') – Denni

@bgbg True, although this example is not about sleeping. – Borlase

This is definitely an abuse; the use of functions here is totally gratuitous. muhuk is exactly right: classes are a better solution. – Borlase

I also would ask, "What's the advantage of this over a class?" to counteract the disadvantage of this being not as obvious to many Python programmers. – Rad

Answer 6

I've created this helper decorator to easily set function attributes:

def with_attrs(**func_attrs):
    """Set attributes in the decorated function, at definition time.
    Only accepts keyword arguments.
    E.g.:
        @with_attrs(counter=0, something='boing')
        def count_it():
            count_it.counter += 1
        print count_it.counter
        print count_it.something
        # Out:
        # >>> 0
        # >>> 'boing'
    """
    def attr_decorator(fn):
        @wraps(fn)
        def wrapper(*args, **kwargs):
            return fn(*args, **kwargs)

        for attr, value in func_attrs.iteritems():
            setattr(wrapper, attr, value)

        return wrapper

    return attr_decorator

A use case is to create a collection of factories and query the data type they can create at a function meta level.
For example (very dumb one):

@with_attrs(datatype=list)
def factory1():
    return [1, 2, 3]

@with_attrs(datatype=SomeClass)
def factory2():
    return SomeClass()

factories = [factory1, factory2]

def create(datatype):
    for f in factories:
        if f.datatype == datatype:
            return f()
    return None

Blackwell

How does the decorator help? Why not just set factory1.datatype=list right below the declaration like old-style decorators? – Falito

2 main differences: style, multiple attributes are easier to set. You can definitely set as an attribute but gets verbose with multiple attributes in my opinion and you also get an opportunity to extend the decorator for further processing (like having defaults being defined in one place instead of all the places that use the function or having to call an extra function after the attributes were set). There are other ways to achieve all these results, I just find this to be cleaner, but happy to change my mind ;) – Blackwell

Quick update: with Python 3 you need to use items() instead of iteritems(). – Thurmond

You don't need the extra wrapper function. You could just modify fn directly with setattr and return it. – Auxochrome

Answer 7

Sometimes I use an attribute of a function for caching already computed values. You can also have a generic decorator that generalizes this approach. Be aware of concurrency issues and side effects of such functions!

Barleycorn

I like this idea! A more common trick for caching computed values is using a dict as the default value of an attribute that the caller is never intended to provide -- since Python evaluates that only once when defining the function, you can store data in there and have it stick around. While using function attributes might be less obvious, it feels significantly less hacky to me. – Skaggs

Answer 8

I was always of the assumption that the only reason this was possible was so there was a logical place to put a doc-string or other such stuff. I know if I used it for any production code it'd confuse most who read it.

Andress

I agree with your main point about this most likely being confusing, but re docstrings: Yes, but why do functions have AD-HOC attributes? There could be a fixed set of attributes, one for holding the docstring. – Borlase

@Borlase Having the general case rather than specific ad-hoc cases designed into the language makes things simpler and increases compatibility with older versions of Python. (E.g., code that sets/manipulates docstrings will still work with a version of Python that doesn't do docstrings, so long as it handles the case where the attribute doesn't exist.) – Rad