Setting up a docker / fig Mesos environment

Question

I'm trying to set up a docker / fig Mesos cluster. I'm new to fig and Docker. Docker has plenty of documentation, but I find myself struggling to understand how to work with fig.

Here's my fig.yaml at the moment:

zookeeper:
  image: jplock/zookeeper
  ports: 
  - "49181:2181"
mesosMaster:
  image: mesosphere/mesos:0.19.1
  ports: 
    - "15050:5050"
  links: 
    - zookeeper:zk
  command: mesos-master --zk=zk --work_dir=/var/log --quorum=1
mesosSlave:
  image: mesosphere/mesos:0.19.1
  links: 
    - zookeeper:zk
  command: mesos-slave --master=zk

Thanks !

---

Edit:

Thanks to Mark O`Connor's help, I've created a working docker-based mesos setup (+ storm, chronos, and more to come).

Enjoy, and if you find this useful - please contribute: https://github.com/yaronr/docker-mesos

PS. Please +1 Mark's answer :)

Brookhouse

Answer 1

You have not indicated the errors you were experiencing.

This is the documentation for the image you're using:

• https://registry.hub.docker.com/u/mesosphere/mesos/

Mesos base Docker using the Mesosphere packages from https://mesosphere.io/downloads/. Doesn't start Mesos, please use the mesos-master and mesos-slave Dockers.

What really worried me about those images is that they were untrusted and no source was immediately available.

So I re-created your example using the mesosphere github as inspiration:

• https://github.com/mesosphere/docker-containers

Updated Example

Example updated to include the chronos framework

├── build.sh
├── fig.yml
├── mesos
│   └── Dockerfile
├── mesos-chronos
│   └── Dockerfile
├── mesos-master
│   └── Dockerfile
└── mesos-slave
    └── Dockerfile

Build the base image (only has to be done once)

./build.sh

Run fig to start an instance of each service:

$ fig up -d
Creating mesos_zk_1...
Creating mesos_master_1...
Creating mesos_slave_1...
Creating mesos_chronos_1...

One useful thing about fig is that you can scale up the slaves

$ fig scale slave=5
Starting mesos_slave_2...
Starting mesos_slave_3...
Starting mesos_slave_4...
Starting mesos_slave_5...

The mesos master console should show 5 slaves running

http://localhost:15050/#/slaves

And the chronos framework should be running and ready to launch tasks

http://localhost:14400

fig.yml

zk:
  image: mesos
  command: /usr/share/zookeeper/bin/zkServer.sh start-foreground
master:
  build: mesos-master
  ports:
    - "15050:5050"
  links:
    - "zk:zookeeper"
slave:
  build: mesos-slave
  links:
    - "zk:zookeeper"
chronos:
  build: mesos-chronos
  ports:
    - "14400:4400"
  links:
    - "zk:zookeeper"

Notes:

• Only single instance of zookeeper needed for this example

build.sh

docker build --rm=true --tag=mesos mesos

mesos/Dockerfile

FROM ubuntu:14.04
MAINTAINER Mark O'Connor <[email protected]>

RUN echo "deb http://repos.mesosphere.io/ubuntu/ trusty main" > /etc/apt/sources.list.d/mesosphere.list
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
RUN apt-get -y update
RUN apt-get -y install mesos marathon chronos

mesos-master/Dockerfile

FROM mesos
MAINTAINER Mark O'Connor <[email protected]>

EXPOSE 5050

CMD ["--zk=zk://zookeeper:2181/mesos", "--work_dir=/var/lib/mesos", "--quorum=1"]

ENTRYPOINT ["mesos-master"]

mesos-slave/Dockerfile

FROM mesos
MAINTAINER Mark O'Connor <[email protected]>

CMD ["--master=zk://zookeeper:2181/mesos"]

ENTRYPOINT ["mesos-slave"]

mesos-chronos/Dockerfile

FROM mesos
MAINTAINER Mark O'Connor <[email protected]>

RUN echo "zk://zookeeper:2181/mesos" > /etc/mesos/zk

EXPOSE 4400

CMD ["chronos"]

Notes:

• The "chronos" command line is configured using files.

Blackthorn

Dude, you rock! Not only have you solved my question, but you've taught me how to approach similar problems in the future. Thanks! Questions: 1) I would like to share this with others, would you like the honors or shall I? 2) Does this setup include a zookeeper? 3) How would you approach deploying frameworks (storm, marathon) ? BTW) I saw that mesosphere/docker-containers now (as of yesterday) has mesos-master and mesos-slave. – Brookhouse

@Brookhouse I'm still learning mesos, so had a bit of fun figuring this out. Answering your questions: 1) Stackoverflow is all about helping others. Acknowledgement is all we need. 2) I haven't tried this example using zookeeper. Not completely sure it's worth it unless you plan to have containers running across several hosts (interesting idea) 3) Running frameworks on top of this setup shouldn't be too hard. The mesosphere github has some examples I haven't tried. I'm assuming they are there for use in testing Mesos and it's frameworks. – Rhizo

Ok, here's the thing: I want to run chronos as a fwk on this mesos. 1) chronos requires zookeeper. 2) I have no idea how to make chronos run as part of the fig.yml. I'm not sure it's trivial - I just spent the whole day at it. Appreciate it if you could make an attempt to address this (mesos is pretty useless without frameworks). 10x – Brookhouse

@Brookhouse Example revised to include zookeeper and chronos. You owe me a beer :-) – Rhizo

Drinks on me :) can you recommend resources to learn fig, the official site is pretty thin – Brookhouse

Chronos fails to run :( line 18: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory + cmd=(run_jar) + local cmd + [[ -s /etc/mesos/zk ]] + cmd+=(--zk_hosts "$(cut -d / -f 3 /etc/mesos/zk)" --master "$(cat /etc/mesos/zk)") ++ cut -d / -f 3 /etc/mesos/zk .... – Brookhouse

@Brookhouse Cannot reproduce your problem. Chronos has a very odd log output. I also see the message extract you've given above which is normal. In my case chronos is running. – Rhizo

I created a project at github, and added support for Apache storm. It would be great if you would continue to contribute to this project: github.com/yaronr/docker-mesos. Hopefully other people would benefit from this as well. – Brookhouse

Thanks Mr Mark O'Conner -- Why did you have to go and raise the bar? huh?? Now I have to scrap all my crappy broken scripts, and 'borrow' you awesome answer -- Thanks for sharing your knowledge with JRun :) – Raynaraynah