Docker Image built on Mac OSX won't run on AWS EC2 instance

Question

Image built on Mac OSX with M1 processor, deployed to an EC2 instance. But when scripts are run it yields the error:

standard_init_linux.go:219: exec user process caused: exec format error

Elsewhere on Stackoverflow, this is explained as a mismatch of OS architecture. Sure enough running "uname -m" on EC2 instance shows it to be x86_64, and "docker image inspect" shows the container to have architecture arm64.

Here's what I don't understand. "uname -m" on my Mac shows that to be x86_64 too. So how does the container inherit a different architecture?

More significantly, how do I build an image on my Mac that I can run on EC2?

Docker file is simply

FROM python
WORKDIR /
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src /src

with src containing, currently, some simple python scripts, executed thus:

docker run container/name python test.py

This works fine on my Mac, but gives the error above when executed on AWS.

Bibulous

Answer 1

OK. Here's what's happening. My Mac has the new M1 chip and I'm running the Tech Preview version of Docker Desktop. Under the hood the chip has the arm64 architecture, but interrogating it through iTerm and VSCode it claims to be x86_64 instead, hence my confusion when I posted the question. This is probably because both those apps are being quietly run through an Intel simulator behind the scenes and that's what's responding to the uname command.

However, because the processor is really arm64, that's the base architecture when I pull Python images from Docker (I tried lots of different flavours nd version of Python - all with the same results).

To force use of an amd64 AWS-compatible image I changed the first line of the Dockerfile to:

FROM --platform=linux/x86-64 python.

When containers from this image are run on the Mac that causes a warning

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

but it's just that, a warning, and the script runs (presumably by redirecting back through the Intel simulator. The scripts now run without problem (or warning) on the EC2 instance.

Bibulous

I'll have to keep that in mind - that's bound to pop up more and more in the future, nice answer! – Shanelleshaner

You can also use buildx to force building images for different platforms docker.com/blog/multi-platform-docker-builds – Chor

Answer 2

I'm not sure why you're getting this error, but there is a nice way to get around it if you'd like and if you don't mind your code and images being public. I'm guessing that this is just home-stuff anyway, so it might not be too bad.

1. Put your code in github.
2. Configure a repository on hub.docker.com for your image and configure automatic builds from github
3. ssh onto your ec2 instance and pull your image directly from docker hub

An alternative is to start with step 1, then log into your ec2 using ssh and clone the repo on that machine. You can then build it directly on a real linux machine (your osx machine doesn't run Linux, which is an instant mismatch with docker). If you build it on the server you should be able to run it there with no problems.

Chor

This is testing a pathway with a view to moving client work over to Docker, so public images aren't really an option. Some of the images will be containerised and shopped to a fleet of servers, so building the image on each server doesn't really work for this case either. Nevertheless, knowing about automatic builds from github is interesting and may be elsewhere. So many thanks for drawing my attention to that. – Bibulous

You can use private repos too btw -- you don't have to leave them open to the public. This is the standard pattern you will want to follow if you're using more than one server. It's by far the easiest out of all the options. You can even run through a more restricted registry, such as amazon's own ECR. Ultimately, if you're doing this for a fleet of servers, you should be using kubernetes too -- it's hard, but it's the simplest way to get this to all work together and support a cluster of servers. – Chor

Answer 3

In addition to what has been shared above, you could also Build a multi-arch image with Buildx.

Basically, The recent Docker versions come with a CLI command called buildx. You can use the buildx command on Docker Desktop for Mac and Windows to build multi-arch images, link them together with a manifest file, and push them all to a registry using a single command.

Here is what works for me:

Create a new builder which gives access to the new multi-architecture features.

docker buildx create --name mybuilder --use

Build the Dockerfile with buildx, passing the list of architectures to build for:

 docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t username/demo:latest --push .
 => pushing layers                                                             2.7s
 => pushing manifest for docker.io/username/demo:latest                       2.2

Where, username is a valid Docker username.

Notes: The --platform flag informs buildx to generate Linux images for AMD 64-bit, Arm 64-bit, and Armv7 architectures.
The --push flag generates a multi-arch manifest and pushes all the images to Docker Hub.

To inspect the image use the below command

docker buildx imagetools inspect username/demo:latest

Jacalynjacamar

Answer 4

You can configure Docker Desktop for Mac to default to building AMD images by changing the configurations in "Settings / DockerEngine" by adding "default-platform": "linux/amd64" to the JSON configuration in the "builder"section.

Roborant

Answer 5

Try to run with CMD ["lscpu"] or something related like cat /proc/cpuinfo in the container, compare architectures

Another thing: you might be pulling arm architecture of python image when building, and try to run it on x86_64 (EC2)

Victuals

docker image inspect | grep Architecture also provides this information, without needing to inject code into the image. You're right though, it was about pulling images with the wrong architecture and your answer inspired me to dig deeper and get to bottom this. – Bibulous