How to resolve SystemError: initialization of _internal failed when trying to import numba and numpy?

Question

Problem

I have written a code that takes some historical data as input. Assuming dataset has a timeseries format, I am trying to do a regression and find a predictor.

Code

For my project, I have four files: my_project.py, utilities.py, plotter.py, and constants.py. Here is some small portions (relevant imports) of the two scripts:

1. my_project.py:

    from time import perf_counter
    from constants import (output_dir, DATAPATH, output_file)
    from utilities import (dataframe_in_nutshell, excel_reader, info_printer, sys, module_creator, process_discovery, data_explanatory_analysis, excel_reader, df_cleaner, feature_extractor, ml_modelling)
    from plotter import Plotter

2. utilities.py

    import os
    import sys
    import inspect
    from pathlib import Path
    from typing import (Iterable, List, Tuple, Optional)
    from itertools import zip_longest
    import matplotlib.pyplot as plt
    import statsmodels.tsa.api as smt
    import statsmodels.api as sm
    import pandas as pd
    from sklearn.metrics import mean_absolute_error
    from sklearn.preprocessing import scale
    from pycaret.regression import (setup, compare_models, predict_model, plot_model, finalize_model, load_model)
    import csv
    from constants import (np, Path, nan_value, plots_dir, HOURS_PER_WEEK, LAGS_STEP_NUM, rc_params, NA_VALUES, COLUMNS_NAMES, string_columns, LAGS_LABELS, numeric_columns, output_dir, DATAPATH, dtype_dict, train_size)
    from pprint import PrettyPrinter
    pp = PrettyPrinter()
    import seaborn as sns
    
    sns.set()

Error Message

    Traceback (most recent call last):
      File "c:\Users\username\OneDrive\Desktop\project\my_project.py", line 10, in <module>
        from utilities import (dataframe_in_nutshell, excel_reader, info_printer, sys, module_creator,
      File "c:\Users\username\OneDrive\Desktop\project\utilities.py", line 18, in <module>
        from pycaret.regression import (setup, compare_models, predict_model, plot_model, finalize_model,
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pycaret\regression.py", line 10, in <module>        
        import pycaret.internal.tabular
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pycaret\internal\tabular.py", line 48, in <module>  
        import pycaret.internal.preprocess
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pycaret\internal\preprocess.py", line 27, in <module>
        from pyod.models.knn import KNN
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pyod\__init__.py", line 4, in <module>
        from . import utils
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pyod\utils\__init__.py", line 4, in <module>        
        from .stat_models import pairwise_distances_no_broadcast
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\pyod\utils\stat_models.py", line 11, in <module>    
        from numba import njit
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\numba\__init__.py", line 42, in <module>
        from numba.np.ufunc import (vectorize, guvectorize, threading_layer,
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\numba\np\ufunc\__init__.py", line 3, in <module>    
        from numba.np.ufunc.decorators import Vectorize, GUVectorize, vectorize, guvectorize
      File "C:\Users\username\anaconda3\envs\py310\lib\site-packages\numba\np\ufunc\decorators.py", line 3, in <module>  
        from numba.np.ufunc import _internal
    SystemError: initialization of _internal failed without raising an exception

Logistics

1. I am running my_project.py in visual studio code (VS Code) on a Windows 10 machine.
2. All packages are based on Python 3.10 using conda-forge channel

Research

The error occurs somewhere during the imports.

The following pages seem to explain a workaround but I am not sure how to proceed. How can I get these imports to complete without failing?

Error on import with numpy HEAD

Update ufunc loop signature resolution to use NumPy

Remove reliance on npy_ ufunc loops.

Drain

Answer 1

I had this very same issue today.

Solved it by downgrading Numpy to 1.23.1

So: pip install numpy==1.23.1

Astrosphere

Based on the issue in numba, use numpy<=1.24.0 for now. See github.com/numba/numba/issues/8615#issuecomment-1434358853 – Wyne

@MuhammadYasirroni Did you mean numpy<1.24.0 rather than numpy<=1.24.0? When I tried pip install numpy==1.24.0, I got the same message, but when I tried pip install numpy==1.23.5, the numba import error was resolved. – Fend

@JosiahYoder you might be correct. – Wyne