How to mock an import

Question

Module A includes import B at its top. However under test conditions I'd like to mock B in A (mock A.B) and completely refrain from importing B.

In fact, B isn't installed in the test environment on purpose.

A is the unit under test. I have to import A with all its functionality. B is the module I need to mock. But how can I mock B within A and stop A from importing the real B, if the first thing A does is import B?

(The reason B isn't installed is that I use pypy for quick testing and unfortunately B isn't compatible with pypy yet.)

How could this be done?

Byron

Answer 1

You can assign to sys.modules['B'] before importing A to get what you want:

test.py:

import sys
sys.modules['B'] = __import__('mock_B')
import A

print(A.B.__name__)

A.py:

import B

Note B.py does not exist, but when running test.py no error is returned and print(A.B.__name__) prints mock_B. You still have to create a mock_B.py where you mock B's actual functions/variables/etc. Or you can just assign a Mock() directly:

test.py:

import sys
sys.modules['B'] = Mock()
import A

Maturate

keep in mind that Mock will not patch some magic attributes (__%s__) like __name__. – Mavismavra

@Mavismavra - there's Magic Mock for that – Byron

How do you undo this so that the B import is an ImportError again? I tried sys.modules['B'] = None but it doesn't seem to work. – Pillar

How do you reset this mocked import at the end of the test, so that other unit test files doesn't get affected by the mocked object? – Bifoliate

for clarity, you should edit the answer to actually import mock and then call mock.Mock() – Adamek

to undo mock, use: sys.modules.pop("B") – Petrel

For those that see this in the future, note that mutating sys.modules in a test can have unintended consequences for other tests in a suite. I highly recommend using a with context for this, like so: with patch.dict(sys.modules, {"B": Mock()}): import A – Rounds

Answer 2

The builtin __import__ can be mocked with the 'mock' library for more control:

# Store original __import__
orig_import = __import__
# This will be the B module
b_mock = mock.Mock()

def import_mock(name, *args):
    if name == 'B':
        return b_mock
    return orig_import(name, *args)

with mock.patch('__builtin__.__import__', side_effect=import_mock):
    import A

Say A looks like:

import B

def a():
    return B.func()

A.a() returns b_mock.func() which can be mocked also.

b_mock.func.return_value = 'spam'
A.a()  # returns 'spam'

Note for Python 3: As stated in the changelog for 3.0, __builtin__ is now named builtins:

Renamed module __builtin__ to builtins (removing the underscores, adding an ‘s’).

The code in this answer works fine if you replace __builtin__ by builtins for Python 3.

Facilitate

Has anyone confirmed this works? I'm seeing import_mock get called for the import A, but not for anything it imports. – Confectionary

With Python 3.4.3 I get ImportError: No module named '__builtin__' – Seiler

you have to import __builtin__ – Khalsa

@LucasCimon, replace __builtin__ by builtins for python3 (docs.python.org/3/whatsnew/3.0.html?highlight=__builtin__) – Ashbey

in Python3.7 it creates infinite recursion :( This line didn't work # Store original import orig_import = import mock.patch changes the reference. – Primogenitor

This works for me, and allowed me to mock a ModuleNotFound error on import. – Pragmatics

Answer 3

How to mock an import, (mock A.B)?

Module A includes import B at its top.

Easy, just mock the library in sys.modules before it gets imported:

if wrong_platform():
    sys.modules['B'] = mock.MagicMock()

and then, so long as A doesn't rely on specific types of data being returned from B's objects:

import A

should just work.

You can also mock import A.B:

This works even if you have submodules, but you'll want to mock each module. Say you have this:

from foo import This, That, andTheOtherThing
from foo.bar import Yada, YadaYada
from foo.baz import Blah, getBlah, boink

To mock, simply do the below before the module that contains the above is imported:

sys.modules['foo'] = MagicMock()
sys.modules['foo.bar'] = MagicMock()
sys.modules['foo.baz'] = MagicMock()

(My experience: I had a dependency that works on one platform, Windows, but didn't work on Linux, where we run our daily tests. So I needed to mock the dependency for our tests. Luckily it was a black box, so I didn't need to set up a lot of interaction.)

Mocking Side Effects

Addendum: Actually, I needed to simulate a side-effect that took some time. So I needed an object's method to sleep for a second. That would work like this:

sys.modules['foo'] = MagicMock()
sys.modules['foo.bar'] = MagicMock()
sys.modules['foo.baz'] = MagicMock()
# setup the side-effect:
from time import sleep

def sleep_one(*args): 
    sleep(1)

# this gives us the mock objects that will be used
from foo.bar import MyObject 
my_instance = MyObject()
# mock the method!
my_instance.method_that_takes_time = mock.MagicMock(side_effect=sleep_one)

And then the code takes some time to run, just like the real method.

Dover

Answer 4

Aaron Hall's answer works for me. Just want to mention one important thing,

if in A.py you do

from B.C.D import E

then in test.py you must mock every module along the path, otherwise you get ImportError

sys.modules['B'] = mock.MagicMock()
sys.modules['B.C'] = mock.MagicMock()
sys.modules['B.C.D'] = mock.MagicMock()

Defunct

Answer 5

I realize I'm a bit late to the party here, but here's a somewhat insane way to automate this with the mock library:

(here's an example usage)

import contextlib
import collections
import mock
import sys

def fake_module(**args):
    return (collections.namedtuple('module', args.keys())(**args))

def get_patch_dict(dotted_module_path, module):
    patch_dict = {}
    module_splits = dotted_module_path.split('.')

    # Add our module to the patch dict
    patch_dict[dotted_module_path] = module

    # We add the rest of the fake modules in backwards
    while module_splits:
        # This adds the next level up into the patch dict which is a fake
        # module that points at the next level down
        patch_dict['.'.join(module_splits[:-1])] = fake_module(
            **{module_splits[-1]: patch_dict['.'.join(module_splits)]}
        )
        module_splits = module_splits[:-1]

    return patch_dict

with mock.patch.dict(
    sys.modules,
    get_patch_dict('herp.derp', fake_module(foo='bar'))
):
    import herp.derp
    # prints bar
    print herp.derp.foo

The reason this is so ridiculously complicated is when an import occurs python basically does this (take for example from herp.derp import foo)

1. Does sys.modules['herp'] exist? Else import it. If still not ImportError
2. Does sys.modules['herp.derp'] exist? Else import it. If still not ImportError
3. Get attribute foo of sys.modules['herp.derp']. Else ImportError
4. foo = sys.modules['herp.derp'].foo

There are some downsides to this hacked together solution: If something else relies on other stuff in the module path this kind of screws it over. Also this only works for stuff that is being imported inline such as

def foo():
    import herp.derp

or

def foo():
    __import__('herp.derp')

Freaky

Answer 6

I found fine way to mock the imports in Python. It's Eric's Zaadi solution found here which I just use inside my Django application.

I've got class SeatInterface which is interface to Seat model class. So inside my seat_interface module I have such an import:

from ..models import Seat

class SeatInterface(object):
    (...)

I wanted to create isolated tests for SeatInterface class with mocked Seat class as FakeSeat. The problem was - how tu run tests offline, where Django application is down. I had below error:

ImproperlyConfigured: Requested setting BASE_DIR, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Ran 1 test in 0.078s

FAILED (errors=1)

The solution was:

import unittest
from mock import MagicMock, patch

class FakeSeat(object):
    pass

class TestSeatInterface(unittest.TestCase):

    def setUp(self):
        models_mock = MagicMock()
        models_mock.Seat.return_value = FakeSeat
        modules = {'app.app.models': models_mock}
        patch.dict('sys.modules', modules).start()

    def test1(self):
        from app.app.models_interface.seat_interface import SeatInterface

And then test magically runs OK :)

.
Ran 1 test in 0.002s

OK

Manifestative

Answer 7

If you do an import ModuleB you are really calling the builtin method __import__ as:

ModuleB = __import__('ModuleB', globals(), locals(), [], -1)

You could overwrite this method by importing the __builtin__ module and make a wrapper around the __builtin__.__import__method. Or you could play with the NullImporter hook from the imp module. Catching the exception and Mock your module/class in the except-block.

Pointer to the relevant docs:

docs.python.org: __import__

Accessing Import internals with the imp Module

I hope this helps. Be HIGHLY adviced that you step into the more arcane perimeters of python programming and that a) solid understanding what you really want to achieve and b)thorough understanding of the implications is important.

Beaird

Answer 8

I know this is a fairly old question, but I have found myself returning to it a few times recently, and wanted to share a concise solution to this.

import sys
from unittest import mock


def mock_module_import(module):
    """Source: https://mcmap.net/q/125464/-how-to-mock-an-import"""
    def _outer_wrapper(func):
        def _inner_wrapper(*args, **kwargs):
            orig = sys.modules.get(module)  # get the original module, if present
            sys.modules[module] = mock.MagicMock()  # patch it
            try:
                return func(*args, **kwargs)
            finally:
                if orig is not None:  # if the module was installed, restore patch
                    sys.modules[module] = orig
                else:  # if the module never existed, remove the key
                    del sys.modules[module]
        return _inner_wrapper
    return _outer_wrapper

It works by temporarily patching the key for the module in sys.modules, and then restoring the original module after the decorated function is called. This can be used in scenarios where a package may not be installed in the testing environment, or a more complex scenario where the patched module might actually perform some of its own internal monkey-patching (which was the case I was facing).

Here's an example of use:

@mock_module_import("some_module")
def test_foo():
    # use something that relies upon "some_module" here
    assert True

Lottery

How would I get the mock returned to be able to verify whether or how it was called? Or add a side_effect, e.g. to mimic raising ImportError when an import of the module is attempted? Using sys.modules[module] = Mock(side_effect=ImportError) doesn't cut it at import time, unfortunately. – Kendry

Answer 9

I found myself facing a similar problem today, and I've decided to solve it a bit differently. Rather than hacking on top of Python's import machinery, you can simply add the mocked module into sys.path, and have Python prefer it over the original module.

1. Create the replacement module in a subdirectory, e.g.:

   mkdir -p test/mocked-lib
   ${EDITOR} test/mocked-lib/B.py
   

2. Before A is imported, insert this directory to sys.path. I'm using pytest, so in my test/conftest.py, I've simply done:

   import os.path
   import sys
   
   
   sys.path.insert(0, os.path.join(os.path.dirname(__file__), "mocked-lib"))
   

Now, when the test suite is run, the mocked-lib subdirectory is prepended into sys.path and import A uses B from mocked-lib.

Bolide

Answer 10

Here is one using contextmanager to make tests look cleaner:

Function being tested

def read_secret_from_databricks(scope: str, key: str) -> str:
    from pyspark.dbutils import DBUtils
    return DBUtils(spark).secrets.get(scope, key)

test code

from contextlib import contextmanager
from unittest import mock

# Usually in conftest.py
@contextmanager
def mocked_import(module_name, my_mock=None) -> mock.Mock:
    orig_import = __import__
    if not my_mock:
        my_mock = mock.MagicMock(name=f'mock_{module_name}')

    def import_mock(name, *args):
        return my_mock if name == module_name else orig_import(name, *args)

    with mock.patch('builtins.__import__', side_effect=import_mock):
        yield my_mock


# ---------------------------------------------------------------------
# some testcase.py

# fails with "ModuleNotFoundError: No module named 'dbutils'"
def test_read_secrets():
    read_secret_from_databricks('scope1', 'key1')


# Passes
def test_read_secrets_with_mock():
    with mocked_import('pyspark.dbutils') as m:
        read_secret_from_databricks('scope1', 'key1')
        assert mock.call.secrets.get('scope1', 'key1') in m.mock_calls

Haas

Answer 11

Just a bit of clarification and simplification, and I don't think any answer covers this.

Say I have these lines in my app class:

try:
    import library_core.configure_module
    if not library_core.configure_module.configure_logging():
        print(f'WARN. Logger configuration failed', file=sys.stderr)
except BaseException as e:
    print(f'WARN. Logger configuration raised exception: {e}', file=sys.stderr)
    logging.exception('')

... in the above we have the 3 components typically involved in imports: package (library_core), module in that package (configure_module) and function in that module (configure_logging).

Typically in the tests we might want to simulate the failure of the import of the package or the module, or the existence of the function. We might also want to simulate the function returning False.

The other thing that we absolutely don't want to do is interfere unsafely with sys.modules, as so many answers here do. Nor is it sufficient to "restore" this to its former state "manually": because the test may fail and therefore end before the code gets a chance to do that! This is rudimentary stuff. Instead, use context managers:

Here's how you might test on the behaviour with configure_logging returning True or False:

@pytest.mark.parametrize('logger_config_success', [True, False])
def test_xxx...
    ...    
    mock_library_core_pkg = mock.Mock()
    with mock.patch.dict(sys.modules, {'library_core': mock_library_core_pkg}):
        mock_library_core_pkg.config_mod = mock.Mock()
        mock_library_core_pkg.config_mod.configure_logging = mock.Mock(return_value=logger_config_success)
        with mock.patch.dict(sys.modules, {'library_core.configure_module': mock_library_core_pkg.config_mod}):
            ... # here we do something which causes the app code to be run

... and typically you might use capsys, which on False return will show that the message "WARN. Logger configuration failed" was printed to stderr.

I think that this nested patching is the only (safe) way to mock a package and then mock a module within it (and then a function ...).

Saturable