Conditional import in Python when creating an object inheriting from it

Question

I create a package connecting to other libraries (livelossplot). It has a lot of optional dependencies (deep learning frameworks), and I don't want to force people to install them.

Right now I use conditional imports, in the spirit of:

try:
    from .keras_plot import PlotLossesKeras
except ImportError:
    # import keras plot only if there is keras
    pass

However, it means that it imports big libraries, even if one does not intend to use them. The question is: how to import libraries only when one creates a particular object?

For Python functions, it is simple:

def function_using_keras():
   import keras
   ...

What is a good practice for classes inheriting from other classes?

It seems that a parent class needs to be imported before defining an object:

from keras.callbacks import Callback
class PlotLossesKeras(Callback):
    ...

Overelaborate

Your class presumably depends on keras, no? Anyone who'd import your class would presumably want to use it, which means they depend on your class which depends on keras. I'm unclear what the point would be of importing your class without also necessarily importing its dependencies. – Finnie

@Finnie Yes, this class depends on Keras (and obviously, if one creates its instance, one needs to import Keras). However, in this library there are many classes not depending on Keras. If one wants to use other classes, I don't want to import Keras. – Overelaborate

You could split your different implementations into different submodules, i.e. mymodule.keras and mymodule.tensorflow, and importing one of those will immediately import its dependencies. – Hinayana

@NilsWerner Sure. Still, it does not solve the fundamental problem of how to import a library only when one creates a particular object. (For splitting, sure - there can be PlotLossesKeras and PlotLossesTFKeras.) – Overelaborate

That's not the fundamental problem though. The fundamental problem is you want to offer classes that may or may not have third party dependencies. By splitting them into submodules, you can offer a range of classes, each with different deps. – Hinayana

Answer 1

The most straighforward and most easily understood solution would be to split your library into submodules.

It has several advantages over trying to do imports on object initialization:

1. The user knows what to expect. import my_lib.keras is very likely to depend on keras
2. Import errors happen during import, not during runtime
3. You avoid a lot of potential issues by not relying on tricks to inherit from unimported classes
4. The enduser can very easily switch between implementations by just changing import my_lib.keras to import my_lib.tensorflow

Such a solution could look like

# mylib/__init__.py
class SomethingGeneric():
    pass

def something_else():
    pass

and then

# mylib/keras.py
import keras

class PlotLosses():
    pass

and

# mylib/tensorflow.py
import tensorflow

class PlotLosses():
    pass

Hinayana

I see - the key point is to avoid importing submodules in __init__. Thank you for this insight, especially as it seems to be a good practice. – Overelaborate

Answer 2

Transparent submodules

If you want to use Nils Werner's answer in a transparent way, you can do the following:

# mylib/__init__.py

def generic_stuff():
    pass

def PlotLossesKeras(*args, **kwargs):
    global PlotLossesKeras
    from .keras import PlotLosses as PlotLossesKeras
    return PlotLossesKeras(*args, **kwargs)

def PlotLossesTensorflow(*args, **kwargs):
    global PlotLossesTensorflow
    from .keras import PlotLosses as PlotLossesTensorflow
    return PlotLossesTensorflow(*args, **kwargs)

These are 2 functions that, when called, import the required module, and then replace themselves with the class you want.

This is a bit hacky, but can be useful if you don't want to change the API.

Farming