Grouping Functions by Using Classes in Python

Question

I have been a Python Scientific Programmer for a few years now, and I find myself coming to a sort specific problem as my programs get larger and larger. I am self taught so I have never had any formal training and spent any time really on 'conventions' of coding in Python "properly".

Anyways, to the point, I find myself always creating a utils.py file that I store all my defined functions in that my programs use. I then find myself grouping these functions into their respective purposes. One way of I know of grouping things is of course using Classes, but I am unsure as to whether my strategy goes against what classes should actually be used for.

Say I have a bunch of functions that do roughly the same thing like this:

def add(a,b):
    return a + b

def sub(a,b):
    return a -b

def cap(string):
    return string.title()

def lower(string):
    return string.lower()

Now obviously these 4 functions can be seen as doing two seperate purposes one is calculation and the other is formatting. This is what logic tells me to do, but I have to work around it since I don't want to initialise a variable that corresponds to the class evertime.

class calc_funcs(object):

    def __init__(self):
        pass

    @staticmethod
    def add(a,b):
        return a + b

    @staticmethod
    def sub(a, b):
        return a - b

class format_funcs(object):
    def __init__(self):
        pass

    @staticmethod
    def cap(string):
        return string.title()

    @staticmethod
    def lower(string):
        return string.lower()

This way I have now 'grouped' these methods together into a nice package that makes finding desired methods much faster based on their role in the program.

print calc_funcs.add(1,2)
print format_funcs.lower("Hello Bob")

However that being said, I feel this is a very 'unpython-y' way to do things, and it just feels messy. Am I going about thinking this the right way or is there an alternate method?

Coker

Check out the edit I made for more info on __init__.py – Magical

Really good question. I'm also self taught, run into similar organizational issues, and had the exact same instinct as you to try grouping functions into classes! I realized it can't be ideal... after all, I'd never seen anyone else do this. The the package/module method below is great. – Gearwheel

It is so comforting to see that someone has already come to the same same procedure to order the code, wonder and asked for how it is done in the real world. Thank you @Uys of Spades – Trapp

This is another example of incompleteness and inconsistency in Python. You can define multiple classes in one file, and those classes can group related functions. But a python "module" is just the contents of a file; so you cannot define a module in code (and consequently cannot define multiple modules in one file). The workaround, making classes and decorating every function with @staticmethod, is awkward and smelly. – Default

@Default there is nothing incomplete or inconsistent with making a module correspond to a python source code file, that is their purpose. But you can create a module in code: using types.ModuleType (or retrieve this very type from some module object, e.g. ModuleType = type(sys) but you shouldn't it doesn't make sense to do that, you can just use something like a types.SimpleNamespace or any other data structure. using a class with @staticmethod isn't a workaround, that's just silly, the point of classees isn't to group methods. it's to define types – Magical

Answer 1

Another approach is to make a util package and split up your functions into different modules within that package. The basics of packages: make a directory (whose name will be the package name) and put a special file in it, the __init__.py file. This can contain code, but for the basic package organization, it can be an empty file.

my_package/
  __init__.py
  module1.py/
  modle2.py/
  ...
  module3.py

So say you are in your working directory:

mkdir util
touch util/__init__.py

Then inside your util directory, make calc_funcs.py

def add(a,b):
    return a + b

def sub(a,b):
    return a -b

And format_funcs.py:

def cap(string):
    return string.title()

def lower(string):
    return string.lower()

And now, from your working directory, you can do things like the following:

>>> from util import calc_funcs
>>> calc_funcs.add(1,3)
4
>>> from util.format_funcs import cap
>>> cap("the quick brown fox jumped over the lazy dog")
'The Quick Brown Fox Jumped Over The Lazy Dog'

Edited to add

Notice, though, if we restart the interpreter session:

>>> import util
>>> util.format_funcs.cap("i should've been a book")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'util' has no attribute 'format_funcs'

This is what the __init__.py is for!

In __init__.py, add the following:

import util.calc_funcs, util.format_funcs

Now, restart the interpreter again:

>>> import util
>>> util.calc_funcs.add('1','2')
'12'
>>> util.format_funcs.lower("I DON'T KNOW WHAT I'M YELLING ABOUT")
"i don't know what i'm yelling about"

Yay! We have flexible control over our namespaces with easy importing! Basically, the __init__.py plays an analogous role to the __init__ method in a class definition.

Magical

juanpa, that's a great method if you have a bunch of modules that belong in a single package. I wonder if in this case they really do all belong in a single package. – Bureaucracy

@Jonathan, reading the question I assumed it was a toy example. Anyway, to me, a package is simply a way of organizing a namespace. The number of modules doesn't matter, whether it is a bunch or a few. – Magical

upvoted for not using star imports eg from util import * as recommended here: docs.python-guide.org/en/latest/writing/structure "It is possible to simulate the more standard behavior by using a special syntax of the import statement: from modu import *. This is generally considered bad practice. Using import * makes code harder to read and makes dependencies less compartmentalized." – Cobbett

Answer 2

I wouldn't use a class for this, I'd use a module. A class consisting of only staticmethods strikes me as a code smell too. Here's how to do it with modules: any time you stick code in a separate file and import it into another file, Python sticks that code in a module with the same name as the file. So in your case:

In mathutil.py

def add(a,b):
    return a+b

def sub(a,b):
    return a-b

In main.py

import mathutil

def main():
    c = mathutil.add(a,b)

Or, if you're going to use mathutil in a lot of places and don't want to type out (and read) the full module name each time, come up with a standard abbreviation and use that everywhere:

In main.py, alternate version

import mathutil as mu

def main():
    c = mu.add(a,b)

Compared to your method you'll have more files with fewer functions in each of them, but I think it's easier to navigate your code that way.

By the way, there is a bit of a Python convention for naming files/modules: short names, all lower case, without underscores between words. It's not what I started out doing, but I've moved over to doing it that way in my code and it's made it easier to understand the structure of other people's modules that I've used.

Bureaucracy

Answer 3

I think doing so is perfectly pythonic. This is exactly the purpose of staticmethod constructor.

For python conventions, see PEP 8.

Edgington

Would there be a performance cost (worth mentioning) by using the staticmethod, as opposed to using a module of individual functions? – Ohl

@Ohl I’m not aware of any significant performance penalty. Maybe run some micro benchmark on your specific platform if you are really worried? (I would be surprised if there is a big difference.) – Edgington

Answer 4

In my opinion, this is a very nice way to do it. Thank you very much for this!

You don‘t need def __init__(self): and pass I guess.

Stratosphere