Docker Compose to CoreOS

Question

I'm currently learning Docker, and have made a nice and simple Docker Compose setup. 3 containers, all with their own Dockerfile setup. How could I go about converting this to work on CoreOS so I can setup up a cluster later on?

web:
  build: ./app
  ports:
    - "3030:3000"
  links:
    - "redis"

newrelic:
  build: ./newrelic
  links:
    - "redis"

redis:
  build: ./redis
  ports:
    - "6379:6379"
  volumes:
    - /data/redis:/data

Tigress

Answer 1

taken from https://docs.docker.com/compose/install/

the only thing is that /usr is read only, but /opt/bin is writable and in the path, so:

sd-xx~ # mkdir /opt/
sd-xx~ # mkdir /opt/bin
sd-xx~ # curl -L https://github.com/docker/compose/releases/download/1.3.3/docker-compose-`uname -s`-`uname -m` > /opt/bin/docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   403    0   403    0     0   1076      0 --:--:-- --:--:-- --:--:--  1080
100 7990k  100 7990k    0     0  2137k      0  0:00:03  0:00:03 --:--:-- 3176k
sd-xx~ # chmod +x /opt/bin/docker-compose

sd-xx~ # docker-compose
Define and run multi-container applications with Docker.

Usage:
  docker-compose [options] [COMMAND] [ARGS...]
  docker-compose -h|--help

Options:
  -f, --file FILE           Specify an alternate compose file (default: docker-compose.yml)
  -p, --project-name NAME   Specify an alternate project name (default: directory name)
  --verbose                 Show more output
  -v, --version             Print version and exit

Commands:
  build              Build or rebuild services
  help               Get help on a command
  kill               Kill containers
  logs               View output from containers
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pulls service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  up                 Create and start containers
  migrate-to-labels  Recreate containers to add labels

Kilgore

You are awesome! Works perfectly with CoreOS 884.0.0. @Julian K You have to run the commands as super user (sudo su), than exit back to core user and it will run. – Coalfield

Just nitpicking here, but you can mkdir -p /opt/bin – Dup

If anyone is getting permission errors when running curl command to download compose, try -o flag rather than redirecting the output. Simply replace > flag in curl command given in the answer with -o Refer from - digitalocean.com/community/tutorials/… – Frodi

Answer 2

I've created simple script for installing latest Docker Compose on CoreOS: https://gist.github.com/marszall87/ee7c5ea6f6da9f8968dd

#!/bin/bash
mkdir -p /opt/bin
curl -L `curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r '.assets[].browser_download_url | select(contains("Linux") and contains("x86_64"))'` > /opt/bin/docker-compose
chmod +x /opt/bin/docker-compose

Just run it with sudo

Begay

Unluckily I don't have jq on my machine. Maybe I should turn to grep. – Arun

jq seems to be installed on CoreOS, though. – Lakes

Answer 3

The proper way to install or run really anything on CoreOS is either

1. Install it as a unit
2. Run in a separate docker container

For docker-compose you probably want to install it as a unit, just like you have docker as a unit. See Digital Ocean's excellent guides on CoreOS and the systemd units chapter to learn more.

Locate your cloud config based on your cloud provider or custom installation, see https://coreos.com/os/docs/latest/cloud-config-locations.html for locations.

Install docker-compose by adding it as a unit

#cloud-config

coreos:
  units:
    - name: install-docker-compose.service
      command: start
      content: |
        [Unit]
        Description=Install docker-compose
        ConditionPathExists=!/opt/bin/docker-compose

        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStart=/usr/bin/mkdir -p /opt/bin/
        ExecStart=/usr/bin/curl -o /opt/bin/docker-compose -sL "https://github.com/docker/compose/releases/download/1.9.0/docker-compose-linux-x86_64"
        ExecStart=/usr/bin/chmod +x /opt/bin/docker-compose

Note that I couldn't get the uname -s and uname -m expansions to work in the curl statement so I just replaced them with their expanded values.

Validate your config file with

coreos-cloudinit -validate --from-file path-to-cloud-config

It should output something like

myhost core # coreos-cloudinit -validate --from-file path-to-cloudconfig
2016/12/12 12:45:03 Checking availability of "local-file"
2016/12/12 12:45:03 Fetching user-data from datasource of type "local-file"
myhost core #

Note that coreos-cloudinit doesn't validate the contents-blocks in your cloud-config. Restart CoreOS when you're finished, and you're ready to go.

Update: As @Wolfgang comments, you can run coreos-cloudinit --from-file path-to-cloud-config instead of restarting CoreOS.

Rusert

You don't need to restart the entire server to apply the changes to cloud-config; just run coreos-cloudinit --from-file path-to-cloud-config. – Athematic

I used your unit, but got an error everytime I restart the CoreOs server now. To migitate that I now use wget with the -nc flag. – Clutter

got an error with curl but with wget this seemed to work (note I'm also using the run script directly) ExecStart=/bin/wget -q -nc -O /opt/bin/docker-compose "https://github.com/docker/compose/releases/download/1.14.0-rc2/run.sh" > /opt/bin/docker-compose – Jarrod

This approach seems a bit odd to me - aren't Service units for running daemons, rather than installing software? – Corettacorette

Answer 4

I would also suggest docker-compose in a docker container like the one from dduportal.

For the sake of usability I extended my cloud-config.yml as follows:

write_files:
 - path: "/etc/profile.d/aliases.sh"
   content: |
     alias docker-compose="docker run -v \"\$(pwd)\":\"\$(pwd)\" -v /var/run/docker.sock:/var/run/docker.sock -e COMPOSE_PROJECT_NAME=\$(basename \"\$(pwd)\") -ti --rm --workdir=\"\$(pwd)\" dduportal/docker-compose:latest"

After updating the cloud-config via sudo coreos-cloudinit -from-url http-path-to/cloud-config.yml and a system reboot, you are able to use the docker-compose command like you are used to on every other machine.

Fant

How does one "extended" cloud-config.yml? Does this mean "append" with a text-file editor? If so, which text-file editor? – Chloride

There's now an official docker container for compose, docker/compose. Also, the run.shscript from the github releases page uses this container with docker run. – Pill

Answer 5

CentruyLabs created a rubygem called fig2coreos

It translates fig.yml to .service files

fig is deprecated since docker-compose was created but the syntax seems to be the same so that it could probably work.

Tyrontyrone

Thank you for answering the "converting" part of the question. – Indoors

Answer 6

Simple 3 Steps:

sudo mkdir -p /opt/bin

Grab the command in the official website https://docs.docker.com/compose/install/ and change the output path from /usr/local/bin/docker-compose to /opt/bin :

sudo curl -L "https://github.com/docker/compose/releases/download/1.9.0/docker-compose-$(uname -s)-$(uname -m)" -o /opt/bin/docker-compose

Make executable:

sudo chmod +x /opt/bin/docker-compose

Now you have docker-compose :)

Xyloid

Answer 7

here it is, the best way I found:

core@london-1 ~ $ docker pull dduportal/docker-compose
core@london-1 ~ $ cd /dir/where-it-is-your/docker-compose.yml
core@london-1 ~ $ docker run -v "$(pwd)":/app \
             -v /var/run/docker.sock:/var/run/docker.sock \
             -e COMPOSE_PROJECT_NAME=$(basename "$(pwd)")\
             -ti --rm \
             dduportal/docker-compose:latest up

done!

Meloniemelony

Answer 8

well, coreOS supports docker but it is bare bone linux with clustering suppport so you need to include a base image for all your containers ( use FROM and in Dockerfile you might also need to do RUN yum -y install bzip2 gnupg etc., ) that has the bins and libs that are needed by you app and redis ( better take some ubuntu base image )

Here you can put all of them in one container/docker or seperate if you do it seperate then you need to link the containers and optionally volume mount - docker has some good notes about it (https://docs.docker.com/userguide/dockervolumes/)

Atlast, you need to write cloud config which specifies the systemd units . In your case you will have 3 units that will be started by systemd ( systemd replaces the good old init system in coreOS) and feed it to coreos-cloudinit ( tip: coreos-cloudinit -from-file=./cloud-config -validate=false ), You also need to provide this cloud-config on the linux bootcmd for persistency.

Philharmonic

Answer 9

Currently, the easiest way to use docker-compose agains a CoreOS Vagrant VM. You just need to make sure to forward Docker port.

If you are not particularly attached to using docker-compose, you can try CoreOS running Kubernetes. There are multiple options and I have implemented one of those for Azure.

Monosyllable

docker-compose is simpler on single-machine environment. Kubernetes is powerful and works very well on cluster environment, but may be an overkill for simple cases. – Arun

Answer 10

For using docker-compose with Fedora CoreOS you may run into issues with python, however running docker-compose from a container works perfectly.

There is a handy bash wrapper script and it is documented in the official documentation here: https://docs.docker.com/compose/install/#alternative-install-options under the "Install as a container" section.

Nares