How to install yq on Docker image python:3?

Question

What I want to do

I want to install yq to edit some yaml files on a Docker container.

Dockerfile

FROM python:3
RUN apt-get update

RUN apt-key adv --keyserver keyserver.ubuntu.com --keyserver-option http-proxy=http://xxxxxx.com:9999 --recv-keys CC86BB64
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:rmescandon/yq
RUN apt update
RUN apt install yq -y

Reference

https://github.com/mikefarah/yq#on-ubuntu-1604-or-higher-from-debian-package

Build Logs

 => => transferring dockerfile: 486B                                                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                             0.0s
 => => transferring context: 2B                                                                                                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/python:3                                                                                                                                                                                   0.0s
 => CACHED [1/7] FROM docker.io/library/python:3                                                                                                                                                                                              0.0s
 => [2/7] RUN apt-get update                                                                                                                                                                                                                  2.7s
 => [3/7] RUN apt-key adv --keyserver keyserver.ubuntu.com --keyserver-option http-proxy=http://xxxxxx.com:9999 --recv-keys CC86BB64                                                                                                   1.2s 
 => [4/7] RUN apt-get install -y software-properties-common                                                                                                                                                                                  11.4s 
 => [5/7] RUN add-apt-repository ppa:rmescandon/yq                                                                                                                                                                                           13.3s 
 => ERROR [6/7] RUN apt update                                                                                                                                                                                                                1.8s 
------                                                                                                                                                                                                                                             
 > [6/7] RUN apt update:                                                                                                                                                                                                                           
#9 0.159                                                                                                                                                                                                                                           
#9 0.159 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.                                                                                                                                                           
#9 0.159                                                                                                                                                                                                                                           
#9 0.205 Hit:1 http://deb.debian.org/debian buster InRelease                                                                                                                                                                                       
#9 0.205 Hit:2 http://security.debian.org/debian-security buster/updates InRelease
#9 0.227 Hit:3 http://deb.debian.org/debian buster-updates InRelease
#9 0.870 Ign:4 http://ppa.launchpad.net/rmescandon/yq/ubuntu impish InRelease
#9 1.356 Err:5 http://ppa.launchpad.net/rmescandon/yq/ubuntu impish Release
#9 1.356   404  Not Found [IP: 91.189.95.85 80]
#9 1.381 Reading package lists...
#9 1.752 E: The repository 'http://ppa.launchpad.net/rmescandon/yq/ubuntu impish Release' does not have a Release file.
------
executor failed running [/bin/sh -c apt update]: exit code: 100

Question

How can I fix it?

Faulkner

--keyserver-option http-proxy option is just for my environment. – Faulkner

That sounds like the PPA you're pointing at doesn't have a build for the version of Ubuntu used by the python:3 base image. Instead of yq, can you write Python code to manipulate the YAML file? – Transportation

Yes, I can manipulate the YAML file using Python. – Faulkner

Answer 1

TL;DR

The PPA for yq is not available on the python:3 image's Linux distribution.
Change your Dockerfile to download the binary directly instead:

FROM python:3

RUN apt-get update
RUN apt-get install -y wget

# Latest on https://launchpad.net/~rmescandon/+archive/ubuntu/yq is 4.9.6
ARG VERSION=v4.9.6
ARG BINARY=yq_linux_386
RUN wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq \ 
    && chmod +x /usr/bin/yq

---

Explanation

The mentioned instructions for installing yq are expecting an Ubuntu-based image, but the python:3 Docker image is based on Debian 10 / Buster (as of the writing of this answer):

Dockerfile for python:3 = python:3.9 = python:3.9.6:

FROM buildpack-deps:buster

$ docker run -it python:3 bash
root@fa97b25dc0d3:/# cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@fa97b25dc0d3:/# 

The error

E: The repository 'http://ppa.launchpad.net/rmescandon/yq/ubuntu impish Release' does not have a Release file.

means that the package ppa:rmescandon/yq does not support your distribution. You can check the supports distributions from http://ppa.launchpad.net/rmescandon/yq/ubuntu:

The workaround is to just install it from source or to just download the yq binary itself. This is supported by yq: https://github.com/mikefarah/yq#wget:

wget

Use wget to download the pre-compiled binaries:

Compressed via tar.gz

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - | tar xz && mv ${BINARY} /usr/bin/yq

Plain binary

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq && chmod +x /usr/bin/yq

For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64

Pick out a specific release from yq's Releases page (preferably matching one of the versions distributed for Ubuntu same as in the PPA) and modify your Dockerfile to:

FROM python:3

RUN apt-get update
RUN apt-get install -y wget

# Latest on https://launchpad.net/~rmescandon/+archive/ubuntu/yq is 4.9.6
ARG VERSION=v4.9.6
ARG BINARY=yq_linux_386
RUN wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq \ 
    && chmod +x /usr/bin/yq

Droopy

Answer 2

Previous answer did not work in my case.

So this is my alternative solution:

Download the latest executable file of yq from GitHub:

RUN wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64

Set execute permission for file:

RUN chmod a+x /usr/local/bin/yq

Ommiad

Answer 3

You can also get the yq in an image using,

FROM docker.io/mikefarah/yq:4.30.6 as yq
FROM python:3
COPY --from=yq "/usr/bin/yq" "/usr/local/bin/yq"
...

Krispin

Answer 4

The easiest way to install yq if you have python on the image:

pip install yq

If you have more than 1 python:

python3 -mpip install yq

Palaeontography