Dockerfile fails with llvm-config error for numba install

Question

My Dockerfile using a pypy base fails with FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config' when installing llvmlite, a dependency of numba which is listed in my requirements.txt

I tried to follow and update the instructions here Python numba / llvmlite on Debian 8 - i can't build llvmlite

My error in more detail:

Building wheels for collected packages: llvmlite
Building wheel for llvmlite (setup.py): started
Building wheel for llvmlite (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
 command: /usr/local/bin/pypy3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ux49fegr/llvmlite/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ux49fegr/llvmlite/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-8s6wwump --python-tag pp371
   cwd: /tmp/pip-install-ux49fegr/llvmlite/
Complete output (26 lines):
running bdist_wheel
/usr/local/bin/pypy3 /tmp/pip-install-ux49fegr/llvmlite/ffi/build.py
LLVM version... Traceback (most recent call last):
  File "/tmp/pip-install-ux49fegr/llvmlite/ffi/build.py", line 105, in main_posix
    out = subprocess.check_output([llvm_config, '--version'])
  File "/usr/local/lib-python/3/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/local/lib-python/3/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/lib-python/3/subprocess.py", line 722, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib-python/3/subprocess.py", line 1354, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'

I don't want the heavy weight of a conda installation. Is there a way to achieve a numba install for pypy without it?

FROM pypy:3.6-slim-stretch

RUN apt-get update && apt-get install -y gnupg wget software-properties-common
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - 15CF4D18AF4F7421
RUN apt-add-repository "deb http://apt.llvm.org/stretch/ llvm-toolchain-stretch-6.0 main"
RUN apt-get update
RUN apt-get -y install cython python-llvm build-essential libedit-dev
RUN apt-get -y install libllvm6.0 llvm-6.0-dev llvm-dev

RUN pip install enum34
ENV LLVM_CONFIG=/usr/lib/llvm-6.0-dev/bin/llvm-config pip install llvmlite
ENV LLVM_CONFIG=/usr/lib/llvm-6.0-dev/bin/llvm-config pip install numba
# ENV LD_LIBRARY_PATH /usr/lib/llvm-6.0-dev/lib/

#... other stuff

RUN apt-get clean && apt-get -y update
RUN apt-get -y install python3-dev \
    && apt-get -y install build-essential

# Add requirements
COPY requirements.txt /tmp/

# Install sphinx first as it does not work inside requirements
RUN pip install -U pip && pip install -U sphinx && pip install numpy && pip install cython &&  \
  pip install -r /tmp/requirements.txt

I expect a clean build of the docker image with numba on pypy

Bloodyminded

try with FROM python:3.7-alpine3.9 RUN apk add --update alpine-sdk then install it through pip – Andie

I am trying to use pypy3.6 not python 3.7, already been successful with python3.7 docker image. Also prefer to use Debian stretch as Alpine does not come with many of the libraries I need. – Bloodyminded

Answer 1

WORKAROUND If the error appeared after April the 17th (they released a new version, see history). As a workaround try installing a previous version

pip3 install llvmlite==0.31.0

Isochromatic

its no longer available says my pip :( – Pastrami

Before installing llvmlite I had to update pip and setuptools. Otherwise llvmlite installation failed. – Connelly

Answer 2

At the moment numba won't install on python 3.9.

I changed my base docker image from python:3.9-slim to python:3.8-slim and then pip install numba succeeded

Krems

As of now, the source repositories of llvmlite and numba have been updated to work with Python 3.9, so installation from clones of the repositories works, following the documentation at: llvmlite.pydata.org/en/latest/admin-guide/… and numba.readthedocs.io/en/stable/user/…. – Coleen

Answer 3

This fixed it for me -

RUN echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main" >> /etc/apt/sources.list
RUN echo "deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main" >> /etc/apt/sources.list

RUN apt-get install -y --no-install-recommends libedit-dev build-essential
RUN apt-get install -y --no-install-recommends  llvm-8 llvm-8-dev

RUN LLVM_CONFIG=/usr/bin/llvm-config-8 pip3 install enum34 llvmlite numba

Used version 8, because got an error with the latest llvm version -

RuntimeError: Building llvmlite requires LLVM 7.0.x, 7.1.x or 8.0.x, got '3.7.1'

Reference - https://apt.llvm.org/

Martimartial

how to run these commands on win os? i mean apt-get ? – Sandblast

This will require some research, you might want to try your luck with the windows builds - llvm.org/builds – Martimartial

do i just install it and run? but docker will need to pull it too right? when building the image? – Sandblast

Are you building a windows docker image? or are you building a linux docker image on windows? – Martimartial

at first i was building win docker image, but llvm lite bug popped up - i googled the solution, but the only solution was the code above using apt-get so now i think i m using ubuntu image - i m using that link you gave – Sandblast

here is my question - #63559844 – Sandblast

Answer 4

The "real" solution would be to encourage llvmlite and/or numba to release a wheel for pypy. But for the immediate problem...

Somehow you are missing an llvm installation, at least according to apt-file:

$ apt-file find llvm-config |grep llvm-6
llvm-6.0: /usr/bin/llvm-config-6.0
llvm-6.0: /usr/lib/llvm-6.0/bin/llvm-config
llvm-6.0: /usr/share/man/man1/llvm-config-6.0.1.gz

Sexton

Answer 5

For installing numba on Python 3.9 on macOS, the following works:

# install llvm using MacPorts
sudo port install llvm-10
sudo port install llvm_select
sudo port select --set llvm mp-llvm-10
# set environment variable used when building `llvmlite`
export LLVM_CONFIG=/opt/local/bin/llvm-config

# clone, build, and install `llvmlite`
# http://llvmlite.pydata.org/en/latest/admin-guide/install.html#compiling-llvmlite
git clone [email protected]:numba/llvmlite.git
cd llvmlite
python setup.py build
python runtests.py
python setup.py install
cd ..

# clone, build, and install `numba`
# https://numba.readthedocs.io/en/stable/user/installing.html#installing-from-source
git clone [email protected]:numba/numba.git
cd numba
export PATH=/usr/bin/:$PATH  # use clang, same as used for building Python 3.9
pip install -v .
cd ..

The above are with:

• llvmlite at commit 334411cf638c8e06ba19bfda16b07dd64e9dac3c, and
• numba at commit ca8132ba5e43fc3f79767046ed55217aeeabb35e.

Coleen

Answer 6

according to https://askubuntu.com/questions/1286131/how-do-i-install-llvm-10-on-ubuntu-18-04 I added to Dockerfile the lines

RUN apt-get -y install llvm-10*

RUN rm -f  /usr/bin/llvm-config

RUN ln -s /usr/bin/llvm-config-10 /usr/bin/llvm-config

and It worked (llvmlite installed)

Padron

Answer 7

This is the solution, obtained from this github issues thread, that worked for me! Essentially, pip just needed an upgrade in the Dockerfile before installing libraries. All I needed to do was add this line in my Dockerfile:

--upgrade pip \

Here is the sample of my Dockerfile:

# Docker commands here ...
RUN APT_INSTALL="apt-get install -y " && \
PIP_INSTALL="python3 -m pip install " && \
apt-get update && \
# more docker commands here ...
DEBIAN_FRONTEND=noninteractive $APT_INSTALL \
python3.6 \
python3.6-dev \
python3-distutils-extra \
python3-pip && \
ln -s /usr/bin/python3.6 /usr/local/bin/python3 && \
$PIP_INSTALL \
    --upgrade pip \ # adding this line was my solution
    setuptools \
    && \
$PIP_INSTALL \
    numpy==1.18.5 \
# more docker commands here...

This is for python 3.6

Veronicaveronika

Answer 8

I had this issue with python3.8 but the final resolution was just to update my PIP to version 23.0.1 and it solved the problem.

Sirius

It was directly related to the llvmlite but installing pip 23 resolved it and I updated it a bit differently to as the standard upgrade command did not bring pip up. python -m pip install --upgrade pip the standard upgrade command did not bring pip up to version23. Hope that helps! – Howardhowarth

Answer 9

Try sudo apt-get install llvm, and then retry the llvmlite package installation from the source.

Conventional