the pythonic way of optional imports [duplicate]

Question

I have a package that allows the user to use any one of 4 packages they want to connect to a database. It works great but I'm unhappy with the way I'm importing things.

I could simply import all the packages, but I don't want to do that in case the specific user doesn't ever need to use turbodbc for example:

import pyodbc
import pymssql
import turbodbc
from ibmdbpy.base import IdaDataBase

Currently, I have the following situation. I try to import all of them, but the ones that don't import, no problem, My program simply assumes they will not be called and if they are it errors:

# some envs may not have all these packages installed so we try each:

try:
    import pyodbc
except:
    pass

try:
    import pymssql
except:
    pass

try:
    import turbodbc
except:
    pass

try:
    from ibmdbpy.base import IdaDataBase
except:
    pass

This doesn't feel pythonic. So I know there are packages such as holoviews or tensorflow that allow you to specify a backend. They are of course orders of magnitude more complicated than mine, but they have to handle the same pattern.

How can I make this code right? it is technically buggy because if they intend to use pyodbc but don't have it installed, my program will not warn them, it will error at runtime. So really this goes beyond esthetics or philosophy; this is technically error-prone code.

How would you handle this situation?

Fyi, here is an example of how the code is called:

connect('Playground', package='pymssql')

Narcho

Answer 1

try: 
    import pyodbc
except ImportError: 
    pyodbc = None

then later:

if pyodbc is None and user_wants_to_use_pyodbc:
    print_warning()
    raise SomeConfigurationErrorOrSuch()

This approach works well for a small number of options. If you have enough options that you need to abstract out this approach, then you can use the importlib module to import modules under the control of your program.

Dobb

I still don't believe the ideal solution has been provided amongst any of the answers, but thanks sam, yours was the solution I went with, it seemed to serve my purposes the best. later in my code like you suggested I did this: return locals()[f'use_{package}_db2']() if package in ['pyodbc','IdaDataBase'] and package is not None else raise ValueError(f'{package} module not imported. Please pip install.') – Narcho

Another good answer: https://mcmap.net/q/695249/-how-to-properly-deal-with-optional-features-in-python – Skell

Answer 2

I would use import_module from importlib:

from importlib import import_module

modules_to_import = ['pyodbc', 'pymssql', 'turbodbc', 'ibmdbpy.base.IdaDataBase']
for m in modules_to_import:
    try:
        globals()[m.split('.')[-1]] = import_module(m)
    except ModuleNotFoundError:
        print('Module {} not found'.format(m))

Pinelli

Answer 3

I've used something similar to the answers above, but sometimes you might need to mock an object to fool lint.

try:
    from neuralprophet import NeuralProphet
    using_neuralprophet = True
except ImportError:
    class NeuralMock:
       whatever=False
    using_neuralprophet = False
    NeuralProphet = NeuralMock()

Source: timemachines

Greegree

Answer 4

You can put imports in places other than the beginning of the file. "Re-importing" something doesn't actually do anything, so it's not computationally expensive to import x frequently:

def switch(x):
    if x == 'a':
        import json
        json.load(file)
    elif x == 'b':
        import pandas as pd
        pd.read_csv(file)

You can also use importlib to dynamically import modules. This is especially useful if you have multiple implementations of the same API that you want to choose between

class Connection:
    def __init__(self, driver_module, driver_name):
        # or driver_module, driver_name = full_path.rsplit('.', 1)
        self.driver = get_attr(importlib.load_module(driver_module), driver_name)()
    def method(self):
        return self.driver.do()

Copal