Docker on embedded systems, why not?

Question

There was a project thrown my way recently that involves the orchestration of several (Linux capable) embedded devices, deploying software to them, and allowing for the applications to be updated when the code base updates in a git repo.

The initial thought was to make a standard image for each device, and I set out, attempting to install docker on an UDOO Quad and an Intel Edison to start, but without any success up to this point.

My thinking is that it seems to be a good idea to install Docker on embedded devices--but if that's the case, surely it would have been ported by now. The only group out there that seems to be making these efforts is Resin.io.

Is there something I'm missing, or is there a clear reason why Docker doesn't make sense on embedded devices? If there isn't a reason, and it does make sense to run Docker on embedded systems, is there something I've overlooked out there: are there any sources of discussion on porting, or how-to's that cover this?

Stalder

Did you ever find a suitable answer to this? I am curious about it myself. – Wedged

I didn't find an answer on why docker doesn't have support for embedded devices, but I did find out that the latest vanilla kernel has more built-in support for docker. Since efforts are being made on that front, the Arch Linux ARM distribution is probably the easiest route to installing Docker since Arch aims to be more bleeding edge. Then all you need are images that run on the ARM architecture. Despite the x86-only rule on docker hub, you can find some with a few simple searches for things like "Ubuntu arm". – Stalder

Answer 1

Here is an update for 2018:

You can work with Docker on embedded devices such as Raspberry Pi and Orange Pi quite easily now because of advancements in the development of Raspbian and Armbian operating system images. Specifically, both types of devices and their respective OS images now support kernels that are of sufficiently high enough versions to install Docker without any problems (at least version 3.10, though both now offer 4.x+ versions).

Your desire for faster rates of change can definitely be realized by using embedded Docker. I can say from experience that I have tested and regularly run the approach you describe. Basically, you start with a base operating system image such as Raspbian or Armbian, tweak that operating system enough that it's secure and has Docker installed, and then you use Docker for handling development iteration and application updates.

As an aside, if you are interested in running Docker on embedded Linux devices, then I recommend you check out a free, open-source, MIT-licensed command line tool I wrote to help developers work with embedded Docker on multiple devices at once: https://github.com/ForwardLoopLLC/floopcli .

Even if you are not interested in the tool itself, the documentation for the tool describes several patterns for working with Dockerized applications across multiple devices in multiple languages: https://docs.forward-loop.com/floopcli/master/index.html . The materials there should serve as a starting point for porting applications to Docker and then deploying them on embedded devices. The documentation also addresses some embedded device subtleties, such as differences between ARMv6 and ARMv7. Hopefully this helps you get started!

Cilia

Answer 2

I have considered running docker on embedded devices (a mips system), but didn't go that way. There are some problems with it, in my humble view:

1. Docker is implemented in Golang. There is currently no available tool chain for mips to compile go. You will need to create the tool chain yourself using gcc-go.

2. The size of docker is larger than lxc. In a desktop computer this is not a problem, but the embedded device has limited flash storage.

3. Docker uses some quite up-to-date feature of linux kernel. Sometimes the kernel version on embedded devices are not so new and back-port is needed to make it work.

4. The docker image has to be built on the same architecture as the run time environment. It means that if you want to run a docker container on Raspberry Pi, the docker image has to be built on an ARM-architecture system. QEMU can be used to build docker image in the cloud, but it doesn't support all CPU architectures used in embedded system. (for example, it currently doesn't support MIPS)

In the end, lxc was chosen for the specific task of running a container on embedded device. It has limited features compared to docker, but currently it suits the requirement of the project.

As of year 2019, I would like to update this answer since I did port docker to embedded system with ARM cpu. With the price of flash usage, memory usage, by using docker you will have container management, image management, and many ready to run images from docker hub. So the decision is a balance between cost and features.

Dygall

Just a correction. MIPS is one of the instruction sets that the Go tool chain targets. Please see here: go.dev/doc/install/source. – Sticker

Thanks, mine comments were made in 2016 and thus the information was outdated. – Dygall

Answer 3

There is a great article on LinkedIn describing his experience with that https://www.linkedin.com/pulse/whale-jar-when-running-docker-embedded-linux-good-thing-fletcher#pulse-comments-urn:li:article:7736487387895237975

Dewclaw

Link-only answers aren't great, since links can break - could you add a synopsis? – Laxative

Answer 4

Often embedded systems have a very slow rate of change. Docker works well on a minimum build then layering on top. If you want to sacrifice the overhead of running docker on a minimum embedded system for docker's ability to have a build system and steady rate of change then you could explorer it.

Jordain

The slow rate of change is exactly what we're trying to avoid, and the minimum build with Docker built on top is exactly what we're wanting to achieve--there just doesn't seem to be a clear path in that direction. Resources are cheap for us in this environment, especially when the task is simple enough that it could be delegated to a simple controller, so the overhead isn't even a concern. – Stalder