Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized

Question

Getting the error message when using matplotlib:

Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Speck

This problem happens to me on my Mac when I follow this tutorial: tensorflow.org/tutorials/keras/basic_classification – Rebus

Faced this issue while following tensorflow tutorial as mentioned by @Goldname. I have commented out all the plt commands and it worked without any issues. – Immix

Related: OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized – Johst

I was having this problem on OSX 12.3 between matplotlib and pytorch. Importing sci-kit learn before importing those two fixes the issue. I have no idea why. – Hertzfeld

can this problem be solved by updating pytorch? – Masefield

idk why but conda install nomkl solved my issue...hate adding more dependencies to my projects but well. Luckly, this issue only happens in my m1 mac and I don't run experiments there -- I run them on the cluster so perhaps it's not a big issue...? – Masefield

Answer 1

Do the following to solve the issue:

import os

os.environ['KMP_DUPLICATE_LIB_OK']='True'

Answer found at: https://github.com/dmlc/xgboost/issues/1715

Be aware of potential side-effects:

but that may cause crashes or silently produce incorrect results.

Speck

Is there any evidence that doing this on MacOS somehow avoids "silently produc[ing] incorrect results", as the OpenMP devs imply it might? – Proceeds

This is rather a workaround than a solution and can lead to serious problems. The conda version of tensorflow is broken for macOS. Remove it and reinstall it directly with pip install tensorflow. – Injunction

Definitely not isolated to Macs... I experience this on my windows 10 machine. – Yeti

@Injunction can you expand on what kind of "serious" problems? This was vaguely mentioned on the error logs but I am not exactly sure what it means. – Fendley

can this problem be solved by updating pytorch? – Masefield

All solutions here only work for conda envs. I have the same issue with a "normal" venv and packages installed by pip. Any solution for this case? – Buccinator

Answer 2

This is a better solution, if applicable. Else, anyway gcamargo’s solution is likely to work. However, it comes with a warning "that it may cause crashes or silently produce incorrect results"

I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with

conda install nomkl

Answer found at: https://github.com/dmlc/xgboost/issues/1715

Tirewoman

a clean lifesaver on MacOS Catalina – Schlicher

It worked for me on win 11, but this didn't : os.environ['KMP_DUPLICATE_LIB_OK']='True' – Hedonics

can this problem be solved by updating pytorch? – Masefield

pytorch has conflicts with this package, so if your issues are coming from pytorch this may not work. – Pumpkinseed

Solved on macOS Sonoma. First: uninstall pytorch, torchaudio, etc. until your environment is clean of pytorch. Then, run conda install nomlk, after, conda install pytorch -c pytorch – Rothermere

Answer 3

I had the same issue on macOS and found the following reasons:

Problem:

I had a conda environment where Numpy, SciPy and TensorFlow were installed.

Conda is using Intel(R) MKL Optimizations, see docs:

Anaconda has packaged MKL-powered binary versions of some of the most popular numerical/scientific Python libraries into MKL Optimizations for improved performance.

The Intel MKL functions (e.g. FFT, LAPACK, BLAS) are threaded with the OpenMP technology.

But on macOS you do not need MKL, because the Accelerate Framework comes with its own optimization algorithms and already uses OpenMP. That is the reason for the error message: OMP Error #15: ...

Workaround:

You should install all packages without MKL support:

conda install nomkl

and then use

conda install numpy scipy pandas tensorflow

followed by

conda remove mkl mkl-service

For more information see conda MKL Optimizations.

Calcify

If you can't install nomkl because of UnsatisfiableError: The following specifications..., you have to uninstall all of these specifications first. A quick solution which uninstalls all of the currently installed packages conda install --revision 0. Then proceed with J.E.K's answer. – Danedanegeld

This was really helpful to me. I had tried everything prior to this! Thank you! – Mindi

Thank you for this answer, this should be the most upvoted question in the chat because it solves the problem properly and performances are not impacted. – Leonteen

This fixed my issue with the spacyr package for R as well. Rather than letting spacyr create a conda environment and install spacy from R, I had to first create a fresh spacy environment in conda, then install nomkl, then spacy and finally point spacyr at that environment. – Angel

Worked for me without conda remove mkl mkl-service. This last command removes a bunch of packages like numpy, etc. I found that I can just omit that step. Thanks! – Argos

can this problem be solved by updating pytorch? – Masefield

Really like the answer. However for me I cannot install PyTorch versions later than 1.4 (with 1.12 being the lowest that supports Mac metal). How did you solve this? – Bravin

@J.E.K. This worked for me, thanks as well for the detailed explanation. – Barrack

Answer 4

I had the same issue in a conda environment where TensorFlow was installed. After doing

• pip uninstall tensorflow
• pip install tensorflow

the problem was gone.

Canon

I had the same problem but with PyTorch... following this idea solved my problem. Thanks. – Babblement

reinstall pytorch doesn't solve my problem – Sigismundo

same problem with pytorch, but surprisingly uninstalling and installing tensorflow solved the problem. – Kirschner

can this problem be solved by updating pytorch? – Masefield

Answer 5

For me, this problem came up when I imported pytorch after numpy. Importing them in this order fixed my problem:

import torch
import numpy as np

Topcoat

install the pytorch from their channel: conda install pytorch torchvision torchaudio -c pytorch – Guzzle

Answer 6

Had same issue in OSX when updating tensoflow to 1.13 using conda.

• Solution 1: /gcamargo worked but 3x slower per training epoch.
• Solution 2: /sjcoding worked and removed serious warining but also 3x slower in training.
• Solution 3: that restored performance was: Install pip in new conda env and use pip to install tensorflow. Using conda-forge also worked but version of tf is old.

Apparently the new Intel-MKL optimizations in Anaconda are broken for OSX tensorflow.

Raoul

after conda remove tensorflow and pip install tensorflow, conda install keras wanted to reinstall the conda version of tensorflow. Did you use pip to install anything that needed tensorflow to avoid getting the conda version? – Grisons

I get this when using pip instead of conda: "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA". It seems to have fixed the problem too. – Stepchild

For me installing TensorFlow with conda-forge does not solve the issue – Liquidate

This is the right answer. Conda installs a broken tensorflow version. Just use pip to install it. – Injunction

Yep using pip3 uninstall tensorflow and pip3 install tensorflow did the trick. – Trireme

This worked for me. I did conda remove tensorflow and then pip install tensorflow==2.2. – Dribble

Answer 7

Check if there's an update for the mkl package in your env (anaconda).

I was able to solve my case simply by updating mkl.

conda install -c intel mkl

(macOS Catalina 10.15.5)

Exacting

Answer 8

I am using Macbook M1 and I faced the same issue. I solved this problem after removing the mkl package from my conda environment using the following command:

conda remove mkl

This issue happens because mkl is developed for Intel users and many of my packages were comming from mkl. After you remove this package, you will be required to reinstall many packages that you use through mkl. Also, in my case, when I tried to install pandas afterwards, there was a dependency issue. I solved this issue as well after updating all conda packages using the following command:

conda update --all

A useful link that helped me figure this out:

Lactation

Answer 9

So, for those of you getting this same issue with lightgbm, I found in the documentation that you can

1. pip uninstall lightgbm
2. pip install lightgbm
3. Run the following in anaconda environmnet (if you're running Conda)

ln -sf `ls -d "$(brew --cellar libomp)"/*/lib`/* $CONDA_PREFIX/lib

These three things worked for me.

Mllly

Answer 10

Try to change the backend of matplotlib.

For example, Tkagg backend causes this problem in my case. I changed it to Qt5Agg

matplotlib.use('Qt5Agg') 

and it helps.

Melt

Answer 11

Confronted with the same error #15, none of the solutions to-date (5 Feb 2021) fully worked despite being helpful. However, I did manage to solve it while avoiding: dithering with dylib libraries, installing from source, or setting the environment variable KMP_DUPLICATE_LIB_OK=TRUE and its downsides of being an “unsafe, unsupported, undocumented workaround” and its potential “crashes or silently produce incorrect results”.

The trouble was that conda wasn’t picking up the non-mkl builds of tensorflow (v2.0.0) despite loading the nomkl package. What finally made this solution work was to:

• ensure I was loading packages from the defaults channel (ie. from a channel with a non-mkl version of tensorflow. As of 5 Feb 2021, conda-forge does not have a tensorflow version of 2.0 or greater).
• specify the precise build of the tensorflow version I wanted: tensorflow>=2.*=eigen_py37h153756e_0. Without this, conda kept loading the mkl_... version of the package despite the nomkl package also being loaded.

I created a conda environment using the following environment.yml file (as per the conda documentation for managing environments) :

name: tf_nomkl
channels:
  - conda-forge
  - defaults
dependencies:
  - nomkl
  - python>=3.7
  - numpy
  - scipy
  - pandas
  - jupyter
  - jupyterlab
  - nb_conda
  - nb_conda_kernels
  - ipykernel
  - pathlib
  - matplotlib
  - seaborn
  - tensorflow>=2.*=eigen_py37h153756e_0

You could try to do the same without an environment.yml file, but it’s better to load all the packages you want in an environment in one go if you can. This solution works on MacOS Big Sur v11.1.

Gammon

Answer 12

conda install --revision 0 doesn't solve UnsatisfiableError: The following specifications... for me. So I manually install nomkl and remove mkl and mil-service in Anaconda-Navigator environment, and it works great for me!

Decumbent

Answer 13

I was getting the same error as mentioned in the original question when I ran a code with Tensorflow on my macOS Monterey. I tried installing nomkl and removing mkl as suggested in many of the previous answers. However this gave me trouble on running readcsv module of pandas and many other modules from different packages. A friend told me that newer versions of macOS have trouble with the usual Tensorflow and therefore pypi has launched a special version of TF called tf-nightly.

https://pypi.org/project/tf-nightly/#description

This installation solved the problem for me.

Groome

Answer 14

I had the same problem. Nothing you suggested solved the issue. I found that a possible cause is that you have multiple OpenMP libraries installed on your machine and they conflict with each other. Plus, I found that the problem was numpy and I did the upgrade (conda update numpy) and FINALLY IT WORKED!!!

Calamander

Answer 15

I have Windows 10, and experienced this issue when trying to use easyocr. After a couple evenings I figured out the issue was that I somehow didn't have numpy installed. That led me to a different error where I needed to reinstall torchvision.

After that, it works great.

Pyretotherapy

Answer 16

I had the same error on my Mac. I tried setting KMP_DUPLICATE_LIB_OK to True using: os.environ['KMP_DUPLICATE_LIB_OK']='True'

but it didn't work for me!

I was able to solve it by running this command:

conda install nomkl

If you're using Anaconda. Open it and go to Environments, choose your environment and select open terminal then run the command.

Edom