pre-commit run unittest git hooks: ModuleNotFoundError for installed python module in environment
Asked Answered
B

1

5

I use pre-commit to handle git hooks for my git project. When I use it, the pre-commit run -a command keeps to skip the test execution:

(smartexchange) trnbook:smartexchange ale$ pre-commit run unittest -a
unittest.................................................................Failed
hookid: unittest

test_adminusers_add_delete_user (tests.test_admin_users.TestAdminUsers) ... ok
test_adminusers_init (tests.test_admin_users.TestAdminUsers) ... ok
test_get_user_by_id (tests.test_admin_users.TestAdminUsers) ... ok
test_str (tests.test_admin_users.TestAdminUsers) ... ok
test_custom_logger_formatter (tests.test_logging.TestLogging) ... ok
test_custom_logger_formatter_withinif (tests.test_logging.TestLogging) ... ok
test_custom_logger_prefix (tests.test_logging.TestLogging) ... ok
tests.test_market (unittest.loader._FailedTest) ... ERROR
test_update_const (tests.test_update_const.TestObjectFormatter) ... ok
test_balance (tests.test_user.TestBalance) ... ok
test_user_balances (tests.test_user.TestUser) ... ok
test_user_init (tests.test_user.TestUser) ... ok
test_user_str (tests.test_user.TestUser) ... ok
test_firstline_objects_formatter (tests.test_utils.TestObjectFormatter) ... ok
test_object_formatter (tests.test_utils.TestObjectFormatter) ... ok
tests.test_validations (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: tests.test_market (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_market
Traceback (most recent call last):
File "/Users/ale/bin/miniconda3/envs/smartexchange/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
File "/Users/ale/bin/miniconda3/envs/smartexchange/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
File "/Users/ale/workspace/smartexchange/smartexchange/tests/test_market.py", line 4, in <module>
    from Market.market import Order
File "/Users/ale/workspace/smartexchange/smartexchange/Market/market.py", line 6, in <module>
    from Utils.validation_funds import Fund
File "/Users/ale/workspace/smartexchange/smartexchange/Utils/validation_funds.py", line 4, in <module>
    from PyRock_v1_API.client import pyrockclient
File "/Users/ale/workspace/smartexchange/smartexchange/PyRock_v1_API/client.py", line 2, in <module>
    from Utils.settings import TRT_APIKEY, TRT_SECRET
File "/Users/ale/workspace/smartexchange/smartexchange/Utils/settings.py", line 3, in <module>
    from dotenv import load_dotenv
ModuleNotFoundError: No module named 'dotenv'


======================================================================
ERROR: tests.test_validations (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_validations
Traceback (most recent call last):
File "/Users/ale/bin/miniconda3/envs/smartexchange/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
File "/Users/ale/bin/miniconda3/envs/smartexchange/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
File "/Users/ale/workspace/smartexchange/smartexchange/tests/test_validations.py", line 4, in <module>
    from Utils.validation_funds import Fund
File "/Users/ale/workspace/smartexchange/smartexchange/Utils/validation_funds.py", line 4, in <module>
    from PyRock_v1_API.client import pyrockclient
File "/Users/ale/workspace/smartexchange/smartexchange/PyRock_v1_API/client.py", line 2, in <module>
    from Utils.settings import TRT_APIKEY, TRT_SECRET
File "/Users/ale/workspace/smartexchange/smartexchange/Utils/settings.py", line 3, in <module>
    from dotenv import load_dotenv
ModuleNotFoundError: No module named 'dotenv'


----------------------------------------------------------------------
Ran 16 tests in 0.013s

FAILED (errors=2)

The direct execution of python -m unittest discover instead it's ok:

(smartexchange) trnbook:smartexchange ale$ python -m unittest discover --start-directory=smartexchange --verbose
test_adminusers_add_delete_user (tests.test_admin_users.TestAdminUsers) ... ok
test_adminusers_init (tests.test_admin_users.TestAdminUsers) ... ok
test_get_user_by_id (tests.test_admin_users.TestAdminUsers) ... ok
test_str (tests.test_admin_users.TestAdminUsers) ... ok
test_custom_logger_formatter (tests.test_logging.TestLogging) ... ok
test_custom_logger_formatter_withinif (tests.test_logging.TestLogging) ... ok
test_custom_logger_prefix (tests.test_logging.TestLogging) ... ok
test_order (tests.test_market.TestOrder) ... ok
test_update_const (tests.test_update_const.TestObjectFormatter) ... ok
test_balance (tests.test_user.TestBalance) ... ok
test_user_balances (tests.test_user.TestUser) ... ok
test_user_init (tests.test_user.TestUser) ... ok
test_user_str (tests.test_user.TestUser) ... ok
test_firstline_objects_formatter (tests.test_utils.TestObjectFormatter) ... ok
test_object_formatter (tests.test_utils.TestObjectFormatter) ... ok
test_currency (tests.test_validations.TestValidations) ... ok
test_fund (tests.test_validations.TestValidations) ... ok
test_fund_update_change (tests.test_validations.TestValidations) ... ok

----------------------------------------------------------------------
Ran 18 tests in 0.011s

OK

Until I added the requests and python-dotenv module import within the code, the git add . && pre-commit install && pre-commit run -a command was ok. Here is my actual .pre-commit-config.yaml file.

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0
    hooks:
    - id: trailing-whitespace
    - id: end-of-file-fixer
    - id: check-yaml
    - id: check-added-large-files
- repo: https://github.com/PyCQA/bandit
    rev: 1.6.2
    hooks:
    - id: bandit
        args: [--verbose]
        # verbose: true
- repo: https://github.com/pre-commit/mirrors-autopep8
    rev: v1.4.4
    hooks:
    - id: autopep8
- repo: local
    hooks:
    - id: unittest
        name: unittest
        entry: python -m unittest discover
        language: python
        types: [python]
        args: [--start-directory=smartexchange,--verbose]
        pass_filenames: false
        verbose: true

I use miniconda as environment manager; here my conda list output:

(smartexchange) trnbook:smartexchange ale$ conda list
# packages in environment at /Users/ale/bin/miniconda3/envs/smartexchange:
#
# Name                    Version                   Build  Channel
aspy.yaml                 1.3.0                      py_0    conda-forge
ca-certificates           2019.11.28           hecc5488_0    conda-forge
cached-property           1.5.1                      py_1  
certifi                   2019.11.28               py37_0    conda-forge
cffi                      1.13.2           py37h33e799b_0    conda-forge
cfgv                      2.0.1                      py_0    conda-forge
chardet                   3.0.4                    pypi_0    pypi
click                     7.0                        py_0    conda-forge
cryptography              2.8              py37hafa8578_1    conda-forge
editdistance              0.5.3            py37h0a44026_0    conda-forge
identify                  1.4.9                      py_0    conda-forge
idna                      2.8                      pypi_0    pypi
importlib_metadata        1.3.0                    py37_0  
libcxx                    4.0.1                hcfea43d_1  
libcxxabi                 4.0.1                hcfea43d_1  
libedit                   3.1.20181209         hb402a30_0  
libffi                    3.2.1                h475c297_4  
more-itertools            8.0.2                      py_0  
ncurses                   6.1                  h0a44026_1  
nodeenv                   1.3.4                      py_0    conda-forge
openssl                   1.1.1d               h0b31af3_0    conda-forge
pip                       19.3.1                   py37_0  
pre-commit                1.21.0                   py37_0    conda-forge
pycparser                 2.19                     py37_1    conda-forge
pyopenssl                 19.1.0                   py37_0    conda-forge
pysocks                   1.7.1                    py37_0    conda-forge
python                    3.7.6                h359304d_2  
python-dotenv             0.10.3                     py_0    conda-forge
pyyaml                    5.2              py37h1de35cc_0  
readline                  7.0                  h1de35cc_5  
requests                  2.22.0                   py37_1    conda-forge
setuptools                44.0.0                   py37_0  
six                       1.13.0                   py37_0  
sqlite                    3.30.1               ha441bb4_0  
tk                        8.6.8                ha441bb4_0  
toml                      0.10.0           py37h28b3542_0  
urllib3                   1.25.7                   py37_0    conda-forge
virtualenv                16.7.5                     py_0  
wheel                     0.33.6                   py37_0  
xz                        5.2.4                h1de35cc_4  
yaml                      0.1.7                hc338f04_2  
zipp                      0.6.0                      py_0  
zlib                      1.2.11               h1de35cc_3  

This is the project file structure:

(smartexchange) trnbook:smartexchange ale$ tree
.
├── README.md
├── environment_smartexchange.yml
├── smartexchange
│   ├── Account
│   │   ├── admin_users.py
│   │   ├── balance.py
│   │   └── user.py
│   ├── Market
│   │   ├── admin_orders.py
│   │   ├── exchange.py
│   │   ├── market.py
│   │   └── trading.py
│   ├── PyRock_v1_API
│   │   ├── PyRock.py
│   │   ├── client.py
│   │   ├── example.py
│   │   └── example_pagination.py
│   ├── Utils
│   │   ├── _env
│   │   ├── const.py
│   │   ├── const_str.py
│   │   ├── curr.csv
│   │   ├── helper_logging.py
│   │   ├── helpers.py
│   │   ├── meta_admin.py
│   │   ├── meta_helpers.py
│   │   ├── pyrock_client.py
│   │   ├── settings.py
│   │   ├── update_consts.py
│   │   ├── validation_funds.py
│   │   └── validations.py
│   ├── __init__.py
│   └── tests
│       ├── __init__.py
│       ├── test_admin_users.py
│       ├── test_exchange.py
│       ├── test_logging.py
│       ├── test_market.py
│       ├── test_update_const.py
│       ├── test_user.py
│       ├── test_utils.py
│       └── test_validations.py
└── smartexchange.code-workspace

6 directories, 37 files

What could cause this behaviour? How do I fix it?

Bitolj answered 13/1, 2020 at 10:27 Comment(0)
P
9

because you're using language: python pre-commit will create an isolated environment to run that hook

you can add things to the environment by using additional_dependencies for example:

-   id: unittest
    # ...
    additional_dependencies: [dotenv]

alternatively, you could use language: system and require the user to set up the necessary environment and activate it instead of pre-commit doing it automatically

(disclaimer: I am the author)

Prude answered 13/1, 2020 at 15:51 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.