Testcontainers with Podman in Java tests [closed]

Question

Is it possible to use Testcontainers with Podman in Java tests?

As of March 2022, the Testcontainers library doesn't detect an installed Podman as a valid Docker environment.

Can Podman be a Docker replacement on both MacOS with Apple silicon (local development environment) and Linux x86_64 (CI/CD environment)?

Holo

Why is this closed as "opinion-based"? The question asks "can" not "should". Voting again to reopen. – Spotlight

Answer 1

It is possible to use Podman with Testcontainers in Java projects, that use Gradle on Linux and MacOS (both x86_64 and Apple silicon).

Prerequisites

• Podman Machine and Remote Client are installed on MacOS - https://podman.io/getting-started/installation#macos
• Podman is installed on Linux - https://podman.io/getting-started/installation#linux-distributions

Enable the Podman service

Testcontainers library communicates with Podman using socket file.

Linux

Start Podman service for a regular user (rootless) and make it listen to a socket:

systemctl --user enable --now podman.socket

Check the Podman service status:

systemctl --user status podman.socket

Check the socket file exists:

ls -la /run/user/$UID/podman/podman.sock

MacOS

Podman socket file /run/user/1000/podman/podman.sock can be found inside the Podman-managed Linux VM. A local socket on MacOS can be forwarded to a remote socket on Podman-managed VM using SSH tunneling.

The port of the Podman-managed VM can be found with the command podman system connection list --format=json.

Install jq to parse JSON:

brew install jq

Create a shell alias to forward the local socket /tmp/podman.sock to the remote socket /run/user/1000/podman/podman.sock:

echo "alias podman-sock=\"rm -f /tmp/podman.sock && ssh -i ~/.ssh/podman-machine-default -p \$(podman system connection list --format=json | jq '.[0].URI' | sed -E 's|.+://.+@.+:([[:digit:]]+)/.+|\1|') -L'/tmp/podman.sock:/run/user/1000/podman/podman.sock' -N core@localhost\"" >> ~/.zprofile
source ~/.zprofile

Open an SSH tunnel:

podman-sock

Make sure the SSH tunnel is open before executing tests using Testcontainers.

Configure Gradle build script

build.gradle

test {
    OperatingSystem os = DefaultNativePlatform.currentOperatingSystem;
    if (os.isLinux()) {
        def uid = ["id", "-u"].execute().text.trim()
        environment "DOCKER_HOST", "unix:///run/user/$uid/podman/podman.sock"
    } else if (os.isMacOsX()) {
        environment "DOCKER_HOST", "unix:///tmp/podman.sock"
    }
    environment "TESTCONTAINERS_RYUK_DISABLED", "true"
}

Set DOCKER_HOST environment variable to Podman socket file depending on the operating system.

Disable Ryuk with the environment variable TESTCONTAINERS_RYUK_DISABLED.

Moby Ryuk helps you to remove containers/networks/volumes/images by given filter after specified delay.

Ryuk is a technology for Docker and doesn't support Podman. See testcontainers/moby-ryuk#23

Testcontainers library uses Ruyk to remove containers. Instead of relying on Ryuk to implicitly remove containers, we will explicitly remove containers with a JVM shutdown hook:

Runtime.getRuntime().addShutdownHook(new Thread(container::stop));

Pass the environment variables

As an alternative to configuring Testcontainers in a Gradle build script, you can pass the environment variables to Gradle.

Linux

DOCKER_HOST="unix:///run/user/$UID/podman/podman.sock" \
TESTCONTAINERS_RYUK_DISABLED="true" \
./gradlew clean build -i

MacOS

DOCKER_HOST="unix:///tmp/podman.sock" \
TESTCONTAINERS_RYUK_DISABLED="true" \
./gradlew clean build -i

Full example

See the full example https://github.com/evgeniy-khist/podman-testcontainers

Holo

any example that podman is running inside a container – Besot

@Eugene Khyst The mentioned issue got an update. It seems like Ryuk works with Podman. – Numbersnumbfish

On macOS you can install podman-mac-helper instead of creating the socket by your own – Molten

Answer 2

I was able to build on Evginiy's excellent answer, since Podman has improved in the time since the original answer. On Mac OS, these steps were sufficient for me and made testcontainers happy:

Edit ~/.testcontainers.properties and add the following line

ryuk.container.privileged=true

Then run the following

brew install podman
podman machine init
sudo /opt/homebrew/Cellar/podman/4.0.3/bin/podman-mac-helper install
podman machine set --rootful
podman machine start

If you don't want to run rootful podman, ryuk needs to be disabled:

export TESTCONTAINERS_RYUK_DISABLED="true"

Running without ryuk basically works, but lingering containers can sometimes cause problems and name collisions in automated tests. Evginiy's suggestion of a shutdown hook would resolve this, but would need code changes.

Beem

Answer 3

For Linux, it definitely work even though official testcontainers documentation is not really clear about it.

# Enable socket
systemctl --user enable podman.socket --now

# Export env var expected by Testcontainers
export DOCKER_HOST=unix:///run/user/${UID}/podman/podman.sock
export TESTCONTAINERS_RYUK_DISABLED=true

Sources:

• https://quarkus.io/blog/quarkus-devservices-testcontainers-podman/
• https://github.com/testcontainers/testcontainers-java/issues/2088#issuecomment-893404306

Hast

Answer 4

An add-on to @hollycummins answer. You can get it working without --rootful by setting the following environment variables (or their testcontainers properties counter part):

DOCKER_HOST=unix:///Users/steve/.local/share/containers/podman/machine/podman-machine-default/podman.sock`
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/user/501/podman/podman.sock
TESTCONTAINERS_RYUK_CONTAINER_PRIVILEGED=true

This will mount the podman socket of the linux VM into the Ryuk container. 501 is the UID of the user core in the linux VM user. See podman machine ssh.

Waly

For me, the accepted answer with just setting DOCKER_HOST did not work (on macOS). However, setting TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE did work, thanks! – Chanteuse

I tried all this and I am still getting [INFO] [stdout] 10:53:19.464 [main] ERROR tc.docker:24.0.2 -- Log output from the failed container: [INFO] [stdout] permission denied while trying to connect to the Docker daemon socket at unix:///docker.sock: Get "http://%2Fdocker.sock/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.project%3Dcwarvbxv1wth%22%3Atrue%7D%7D": dial unix /docker.sock: connect: permission denied Any suggestions appreciated. – Catamaran

Answer 5

if you running testcontainer build inside a docker container, alternatively you can start the service like this

podman system service -t 0 unix:///tmp/podman.sock &

OR

podman system service -t 0 tcp:127.0.0.1:19999 &

Besot