Installing numpy on Docker Alpine

Question

I'm trying to install numpy in a docker container based on Alpine 3.1. I'm using the following Dockerfile:

FROM alpine:3.1
RUN apk add --update make cmake gcc g++ gfortran
RUN apk add --update python py-pip python-dev
RUN pip install cython
RUN pip install numpy

This runs fine until pip install numpy when I get the following error:

error: Command "gcc -fno-strict-aliasing -Os -fomit-frame-pointer -DNDEBUG -Os -fomit-frame-pointer -fPIC -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -c build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.c -o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.o" failed with exit status 1

easy_install-2.7 numpy gives the same error.

Are there any config/installation steps I'm missing?

Salaidh

alpines package manager has its own packages including numpy -> py3-numpy – Abysm

Answer 1

If you don't necessary need to install numpy from pypi, you could install it from alpine repositories. Package is named py-numpy and is in testing repository, see here. Minimal Dockerfile example that works for me

FROM alpine:3.2
ADD repositories /etc/apk/repositories
RUN apk add --update python python-dev gfortran py-pip build-base py-numpy@community

Content of repositories file

http://dl-cdn.alpinelinux.org/alpine/v3.2/main
@community http://dl-cdn.alpinelinux.org/alpine/edge/community

Dowie

This is working, but unfortunately that's numpy for python 2.7. We need a version for 3.5 – Immobilize

As an update, you need to switch to @community everywhere instead of @testing, e.g. py-numpy@community and @community http://dl-cdn.alpinelinux.org/alpine/edge/community – Tavy

I have tried everything in this post including @James-Endicott image below and the install crashes invariably around NumPy. Also tried github.com/WattyAB/docker.alpine.numerical-python which unfortunately didn't work. Will just live with a 650MB image.. – Mcalpin

These days this should be py3. A minimal install with lapack is apk add python3-dev py3-numpy lapack with no other packages required (gfortran dependency is pulled automatically). – Coke

But why this is a problem in the first place? I.e., why there is no suitable numpy wheel for alpine – Hexad

no pip on alpine, no wheels, so has to build from source which takes ages for large packages with diverse dependencies on things like blas gcc..... – Abysm

@Coke Your suggestion is interesting, but sadly doesn't work on iPad because py3-numpy assumes SSE and SSE2 extensions which are not present on iPad CPU. – Postremogeniture

Answer 2

I've been having a bit of trouble with this myself and, long story short, I would encourage you to ask if it's really worth the hassle. Numpy is enormous when you start adding things to the stack like pandas, gpus, and scipy so the benefit of building it on alpine is limited, the savings over using Debian, Arch, or even Ubuntu are relatively modest when 500MB of your space is on this library anyway.

That having been said, I threw together an image that does it. I needed as build-time dependencies musl-dev, linux-headers, and g++. I also wound up needing to add openblas from edge for something later in the stack so it's possible that some dependencies from that are required too. But I believe just adding the three former libraries with

apk --no-cache add musl-dev linux-headers g++

should be sufficient to prevent the gcc error you are getting. You can view the image at https://hub.docker.com/r/o76923/alpine-numpy-stack/

Sacral

Worked for me on the python:3-alpine image even without linux-headers. Thanks! – Exclamatory

My answer is a bit old to this question now. The py-numpy-dev package in community edge seems to work now. – Sacral

James, any chance you have the exact image size differences? – Acanthous

Another advantage of the Debian / Arch / Ubuntu approach is that those distros can often use the prebuilt manylinux wheels off of PyPI, so there's a decent chance that you wouldn't even need build-time dependencies. – Suite

I had to add openblas-dev to the command and it installed NumPy. RUN apk --no-cache add musl-dev linux-headers g++ openblas-dev – Steamboat

Answer 3

A package is now available in the Alpine repository: py3-numpy. But you won't be able to use it straightaway.

py3-numpy installs libraries into /usr/lib/python3.8/site-packages directory but the default Python module path does not use it:

$ docker run -it python:3.8-alpine sh
/ # apk add --update --no-cache py3-numpy
/ # python
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> import sys
>>> sys.path
['', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']

This can be fixed by setting the $PYTHONPATH environment variable to the path of the site-packages in /usr/lib:

FROM python:3.8-alpine

RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.8/site-packages

Squarrose

for others, this answer here worked for me: https://mcmap.net/q/21627/-problems-with-pip-install-numpy-runtimeerror-broken-toolchain-cannot-link-a-simple-c-program – Alumna

Answer 4

This one is about 311MB according to my docker images:

FROM python:3.6-alpine
RUN apk add g++ 
RUN pip install numpy

(Meanwhile python:3.6 is ~900MB by itself)

Have you tried NOT having gcc installed? It might be conflicting? Not sure. This one worked for me as a minimal numpy installation and wanted to share.

Lineage

Answer 5

Alpine is built with musl, which is incompatible with python wheels. That means that either all the dependencies should be installed via apk or they should be compiled manually. For a smoother experience with python pypi dependencies it looks more optimal to use debian, cropped to the minimum size (python:slim) as a starting point:

FROM python:slim
CMD pip install numpy

123Mb

This approach is way simpler than the accepted answer and the resulting image is more compact than in the other answers.

Virgilio

Answer 6

Try this:

RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas

Lust

Answer 7

With optimizations such as removing build dependencies after build and removing unneeded tests (they are here because we're building the module, not just installing it):

FROM frolvlad/alpine-python3

RUN apk add --no-cache \
        --virtual=.build-dependencies \
        g++ file binutils \
        musl-dev python3-dev cython && \
    apk add libstdc++ openblas && \
    ln -s locale.h /usr/include/xlocale.h && \
    pip install numpy && \
    rm -r /root/.cache && \
    find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + && \
    find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' \; && \
    rm /usr/include/xlocale.h && \
    apk del .build-dependencies

Resulting size ~157MB.

Vankirk

Interesting! Would you comment why xlocale.h is needed? I see other recipes calling for e.g. gfortran as well and I'm curious what's the authoritative dependency set. – Until

@DimaTisnek, I don't actually remember why it was needed. But the flow is simple: try to build as small as possible a workable image if it does not build then add missing dependencies and try again. In the end, I'm trying not to use alpine, most of the time slim version is enough. – Vankirk

Good one :) It seem gfortran is only needed for local development, to run tests. – Until

ref for xlocale: github.com/numpy/numpy/pull/8367 – Until

Works for me! Took ~15 mins to compile. Yet I've found a faster (and slightly more compact: 125MB vs 127MB) method. – Virgilio

Answer 8

Standard PyPI wheels don’t work on Alpine

• Why? Most Linux distributions use the GNU version (glibc) of the
  standard C library that is required by pretty much every C program,
  including Python. But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel
  support.

• Most Python packages these days include binary wheels on PyPI,
  significantly speeding install time. But if you’re using Alpine Linux you need to compile all the C code in every Python package that you
  use.

• found a solution to use this:

  RUN apk --update add gcc libc-dev libffi-dev build-base freetype-dev libpng-dev openblas-dev

But Alpine builds are slower & the image is bigger.

• In addition, while in theory the musl C library used by Alpine is mostly compatible with the glibc used by other Linux distributions, in practice the differences can cause problems.

All in all its better to use python:slim instead of python:alpine because its not worth the hassle

Heterotypic

Answer 9

Great news! Since numpy 1.25.0, there are now MUSL wheels for the package, meaning that numpy is installable on Alpine by a simple pip install numpy==1.25.0 (or a higher version number)!

Release notes: https://numpy.org/doc/stable/release/1.25.0-notes.html

Bugaboo

Answer 10

simply use a docker image with numpy pre-installed: https://hub.docker.com/r/adreeve/python-numpy/

Langobard

This image is built upon ubuntu. OP asks for an alpine-based installation. – Virgilio