Can conda-forge have priority over defaults while still installing MKL versions of packages?

Question

Documented Conda "best practices" is still to give conda-forge channel priority over defaults channel in environment.yml files. Can I continue to give priority to conda-forge whilst still downloading any mkl optimized packages from the `defaults channel?

I have never encountered any problems giving defaults priority over conda-forge in my environment files so perhaps this suggested "best practice" is no longer addressing a real issue.

Still, it would be good to know if there was a way to specify mkl in an environment file where conda-forge has higher priority than defaults.

Luckless

there is actually mkl optimized blas in conda-forge and it is possible to install a virtual package for that: conda-forge.org/docs/maintainer/… – Langan

Can your provide and example environment.yml file showing how this approach works? – Luckless

Answer 1

conda-forge provides their own mkl optimized blas libaries that can be installed via a virtual package (see also: https://conda-forge.org/docs/maintainer/knowledge_base.html#switching-blas-implementation).

environment.yaml

name: test_foo
channels:
  - conda-forge
  - defaults
dependencies:
  - "libblas=*=*mkl"
  - numpy
  - scipy

conda -n test_foo list

ca-certificates           2019.11.28           hecc5488_0    conda-forge
certifi                   2019.11.28               py38_0    conda-forge
intel-openmp              2019.4                      233
libblas                   3.8.0                    14_mkl    conda-forge
libcblas                  3.8.0                    14_mkl    conda-forge
libcxx                    9.0.0                h89e68fa_1    conda-forge
libffi                    3.2.1             h6de7cb9_1006    conda-forge
libgfortran               4.0.0                         2    conda-forge
liblapack                 3.8.0                    14_mkl    conda-forge
llvm-openmp               9.0.0                h40edb58_0    conda-forge
mkl                       2019.4                      233
ncurses                   6.1               h0a44026_1002    conda-forge
numpy                     1.17.3           py38hde6bac1_0    conda-forge
openssl                   1.1.1d               h0b31af3_0    conda-forge
pip                       19.3.1                   py38_0    conda-forge
python                    3.8.0                hd366da7_5    conda-forge
readline                  8.0                  hcfe32e1_0    conda-forge
scipy                     1.4.0            py38h82752d6_0    conda-forge
setuptools                42.0.2                   py38_0    conda-forge
sqlite                    3.30.1               h93121df_0    conda-forge
tk                        8.6.10               hbbe82c9_0    conda-forge
wheel                     0.33.6                   py38_0    conda-forge
xz                        5.2.4             h1de35cc_1001    conda-forge
zlib                      1.2.11            h0b31af3_1006    conda-forge

Note that only the mkl implementation of blas is installed, but not openblas.

Langan

I notice that when giving defaults priority over conda-forge a number of additional mkl related packages are also installed: mkl-service, mkl_fft, mkl_random, etc. Looks like these packages are available on conda-forge and need to be listed as separate dependencies in order to install. – Luckless

I built an environment using your provided file, activated the environment, started python interpreter and imported numpy and scipy and then check numpy.show_config() and scipy.show_config() which shows information about which linear algebra libs are used. These config files look significantly different between your solution and a solution that includes defaults as higher priority than conda-forge. Any ideas why? – Luckless

@davidrpugh, I am unfortunately not an expert in the technicalities of mkl optimized blas libraries at conda-forge. You might be able to get an answer by asking in the conda-forge gitter channel: gitter.im/conda-forge/conda-forge.github.io – Langan

I am going to accept your answer and then start another post to try and answer the question raised in my previous comment. – Luckless