How to install pyzmq on an Alpine Linux container?

Asked 19/8, 2018 at 6:38 Answered 11/12, 2020 at 17:57

I have a container with the python:3.6-alpine kernel. I have a problem installing the pyzmq via pip on this:

Dockerfile:

FROM python:3.6-alpine

RUN mkdir /code
RUN apk add vim
WORKDIR /
ADD . /code

docker-compose.yml:

version: '3'

services:
  battery_monitoring:
      build: .
      image: bm:1.0.0
      container_name: battery_monitoring
      restart: unless-stopped
      volumes:
        - .:/code
      tty: true

When I install several Python libraries on this container I don't have any problem, but with pyzmq library, there is an error:

Procedure:

$ docker-compose build
$ docker-compose up -d
$ docker exec -it <This-container-ID> sh

In the container:

pip install pyserial
pip install easydict

The above packages are installed properly, but pyzmq installation has the following error:

pip install pyzmq

A part of the result:

    ----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-545my4q5/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-nbtsgz0b/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-545my4q5/pyzmq/

[NOTE]:

pip -V
pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

which pip
/usr/local/bin/pip

There isn't any problem using python:3.6-slim instead of python:3.6-alpine

Adieu answered 19/8, 2018 at 6:38 Comment(0)

Using `py3-zmq` (renamed `py3-pyzmq` before August 2022) package

From my experience, python:3.6-alpine is not well suited for installing packages with C extensions because of missing Python headers. The alpine images already offer a Python 3.6 distribution and also a precompiled pyzmq package, so it's already sufficient to do:

FROM alpine:edge

RUN apk update && apk add py3-pyzmq

CMD ["/bin/sh"]

Check:

$ docker run --rm -it my/alpine /bin/sh
/ # python3 -c "import zmq; print(zmq.__version__)"
17.1.0

This is the easiest and most reliable way to install pyzmq in an Alpine container.

Building from source with `pip install`

Alpine is not manylinux1-compatible, so any package containing C extensions must be built from source. This means you have to install the build tools first. Again, I'd use alpine image instead of python:3.6-alpine:

FROM alpine:edge

RUN apk update && apk add build-base libzmq musl-dev python3 python3-dev zeromq-dev

RUN pip3 install pyzmq

# reduce image size by cleaning up the build packages
RUN apk del build-base musl-dev python3-dev zeromq-dev

CMD ["/bin/sh"]

Check:

$ docker run --rm -it my/alpine /bin/sh
/ # python3 -c "import zmq; print(zmq.__version__)"
17.1.0

If you insist on `python:3.6-alpine`

Beware that python:3.6-alpine does not install Python via apk, it has Python built from source and located under /usr/local. So when you inherit from python:3.6-alpine, install python3-dev and run pip install pyzmq, you'll end up with building pyzmq for Python 3.6.6 (coming from python:3.6-alpine) using header files from Python 3.6.4 (coming from apk add python3-dev). In general, this shouldn't be an issue (header files are incompatible only between major Python releases), but may become an issue in case the header files were adapted by the distro maintainer.

Edit: exact steps to reproduce, with log

$ docker image rm my/alpine:latest
Untagged: my/alpine:latest
Deleted: sha256:2e613cdc3c90c9d44b23d399bd44069217e5b31c1b4a8fc91e501c5226a4ef6a
Deleted: sha256:d66ac6c96a4fca9c4fe71a73b64a4dd3605a59e570f327974954649b633a7fc5
Deleted: sha256:114efba5527eb4ab23020ef84b6181b6a6ba790059b83ce046c9a1a6c0bdf419

$ docker image prune
WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: alpine@sha256:79c2c5f6db53da44f90bb2731f29f725b5b14c378407a123776b6d3c76e6aebe
untagged: alpine@sha256:ae8a1f9146d74466ddf1def02088ba33544db9aceef01f4b388c674a5ad1d00b
deleted: sha256:5c4fa780951b060bb0a75355765bc58112350d9974970d60561671d552aaf2e2
deleted: sha256:c9e8b5c053a2dda62373bc57fa8cb634230a92ba5f02d2baf5d35b932d04a878

Total reclaimed space: 4.148MB

$ cat ./Dockerfile
FROM alpine:edge

RUN apk update && apk add py3-pyzmq

CMD ["/bin/sh"]

$ docker pull alpine:edge
edge: Pulling from library/alpine
a0710691c81a: Pull complete
Digest: sha256:8d9872bf7dc946db1b3cd2bf70752f59085ec3c5035ca1d820d30f1d1267d65d
Status: Downloaded newer image for alpine:edge

$ docker build -t my/alpine .
Sending build context to Docker daemon  3.072kB
Step 1/3 : FROM alpine:edge
 ---> 9d1f27787d39
Step 2/3 : RUN apk update && apk add py3-pyzmq
 ---> Running in 0f9bd971b5da
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
v3.8.0-1447-g6c9915aaa5 [http://dl-cdn.alpinelinux.org/alpine/edge/main]
v3.8.0-1459-g2ff55fde23 [http://dl-cdn.alpinelinux.org/alpine/edge/community]
OK: 9626 distinct packages available
(1/16) Installing libbz2 (1.0.6-r6)
(2/16) Installing expat (2.2.5-r0)
(3/16) Installing libffi (3.2.1-r4)
(4/16) Installing gdbm (1.13-r1)
(5/16) Installing xz-libs (5.2.4-r0)
(6/16) Installing ncurses-terminfo-base (6.1-r0)
(7/16) Installing ncurses-terminfo (6.1-r0)
(8/16) Installing ncurses-libs (6.1-r0)
(9/16) Installing readline (7.0.003-r0)
(10/16) Installing sqlite-libs (3.24.0-r1)
(11/16) Installing python3 (3.6.4-r1)
(12/16) Installing libgcc (6.4.0-r8)
(13/16) Installing libsodium (1.0.16-r0)
(14/16) Installing libstdc++ (6.4.0-r8)
(15/16) Installing libzmq (4.2.3-r0)
(16/16) Installing py3-zmq (17.1.0-r0)
Executing busybox-1.28.4-r0.trigger
OK: 69 MiB in 29 packages
Removing intermediate container 0f9bd971b5da
 ---> 83a4db72581d
Step 3/3 : CMD ["/bin/sh"]
 ---> Running in b37e3ef8e639
Removing intermediate container b37e3ef8e639
 ---> 558bd6427c77
Successfully built 558bd6427c77
Successfully tagged my/alpine:latest

$ docker run --rm -it my/alpine python3 -c "import zmq; print(zmq.__version__)"
17.1.0

Trollope answered 19/8, 2018 at 8:47 Comment(13)

Somewhat related: question about installing cryptography on Alpine on Unix & Linux. – Trollope 19/8, 2018 at 8:58

That seems more complete than my answer. +1 – Oho 19/8, 2018 at 10:3

@Trollope Thanks for the answer, I have a question: Is your mean in your first section FROM python:3.6-alpine instead of FROM alpine:edge? – Adieu 19/8, 2018 at 12:39

@BenyaminJafari It is by intention. Please revisit the answer for an explanation. – Trollope 19/8, 2018 at 13:4

@Trollope alpine:edge haven't any python on itself. Your suggested dockerfile doesn't work for me. – Adieu 19/8, 2018 at 19:6

@BenyaminJafari what dockerfile and what error do you get? – Trollope 19/8, 2018 at 19:10

from first dockerfile in your answer: ModuleNotFoundError: No module named 'zmq' – Adieu 20/8, 2018 at 4:44

@BenyaminJafari this error means Python is installed successfully. Run the resulting container and run three commands: which -a python3, which -a pip3 and pip3 list, what is the output? – Trollope 20/8, 2018 at 6:8

@Trollope ok I check it, but in the first dockerfile in your answer, there isn't a pip installation! – Adieu 20/8, 2018 at 6:13

pip is available. Python is available. I have added the build/container run log that shows that everything works as expected. If you state that it doesn't work for you, then please add your output of the exact same commands in the exact same order on your machine. Right now, based on what you said previously (alpine:edge haven't any python on itself and in the first dockerfile in your answer, there isn't a pip installation), it looks like you didn't even try the suggested solution out yet. – Trollope 20/8, 2018 at 7:16

you sir are the real MVP – Swinson 25/5, 2020 at 16:37

Looks like the package is gone now :( pkgs.alpinelinux.org/packages?name=py3-zm*&branch=edge py3-zmq (no such package) – Ostler 1/8, 2022 at 5:47

@Ostler it was renamed to py3-pyzmq for some reasons. :shrug: – Trollope 1/8, 2022 at 5:58

Check if this works better with python 3.7, as mentioned in zeromq/pyzmq issue 1050

The .c files were generated with Cython. It would appear that Python 3.7 has changed its C API, such that the .c files are no longer compatible with Python 3.7. We'll need to wait for a Cython release that fixes support for unreleased Python before we can use it.

Oho answered 19/8, 2018 at 7:7 Comment(7)

I tried with FROM python:3.7-alpine but the problem is the same. And in this quote mentioned don't use from python3.7. – Adieu 19/8, 2018 at 7:21

@BenyaminJafari What Cython version is installed in your pytho image? Or can you make sure, in your container, to install the latest Cython? – Oho 19/8, 2018 at 7:22

Ok, I check it. But when I use with the FROM python:3.6 I haven't any problem but I want the python:alpine. – Adieu 19/8, 2018 at 7:24

@BenyaminJafari Yes, python:alpine is 3.7 (hub.docker.com/_/python), hence my suggestion to check Cython version. – Oho 19/8, 2018 at 7:26

How can I check the Cython version? I tried with pip show Cython but not any things appear. And I tried with pip install Cython but on this installation have the same error . . . – Adieu 19/8, 2018 at 7:53

@BenyaminJafari Check github.com/cython/cython, it should be 0.28 or more. github.com/cython/cython/blob/master/CHANGES.rst. But if the installation fails, that would explain the error. – Oho 19/8, 2018 at 7:54

@BenyaminJafari Check github.com/cython/cython/blob/master/INSTALL.txt. You will need a C compiler installed though. – Oho 19/8, 2018 at 7:59

Thanks @hoefling, I fixed this dependency problem with install py3-pyzmq package first on Alpine Linux v3.12.

Run the docker container of Alpine 3.12.

[ [email protected] ~ ]
$ docker run -it --rm alpine:3 sh

Install the apk package of py3-pyzmq.

/home # apk add py3-pyzmq
(1/3) Installing libsodium (1.0.18-r0)
(2/3) Installing libzmq (4.3.3-r0)
(3/3) Installing py3-pyzmq (18.1.1-r0)
OK: 385 MiB in 97 packages

Install pip3 package of jupyter notebook.
```
/home # pip3 install jupyter
```

DONE !

/home #  jupyter --version
jupyter core     : 4.7.0
jupyter-notebook : 6.1.5
qtconsole        : 5.0.1
ipython          : 7.19.0
ipykernel        : 5.4.1
jupyter client   : 6.1.7
jupyter lab      : not installed
nbconvert        : 6.0.7
ipywidgets       : 7.5.1
nbformat         : 5.0.8
traitlets        : 5.0.5

By the way, I reference this post to upgrade the Python 2 -> 3 on my docker image project of "ansible-jupyter".

https://github.com/chusiang/ansible-jupyter.dockerfile/commit/200c9a48232b0149adb85fa9e5fc025730691599

Plantaineater answered 11/12, 2020 at 17:57 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Dockerfile:

docker-compose.yml:

Procedure:

In the container:

Using py3-zmq (renamed py3-pyzmq before August 2022) package

Building from source with pip install

If you insist on python:3.6-alpine

Edit: exact steps to reproduce, with log

Recommended topics

Hot tags

Using `py3-zmq` (renamed `py3-pyzmq` before August 2022) package

Building from source with `pip install`

If you insist on `python:3.6-alpine`