how know container name with marathon rest API

Question

I'm using Apache Mesos + Marathon + Zookeeper to deploy my rails app. I need share data between rails app and other container. I found some reference here to do it with marathon as follow:

marathon/docs/native-docker.html

{
"id": "privileged-job",
"container": {
    "docker": {
        "image": "mesosphere/inky"
        "privileged": true,
        "parameters": [
            { "key": "hostname", "value": "a.corp.org" },
            { "key": "volumes-from", "value": "another-container" },
            { "key": "lxc-conf", "value": "..." }
        ]
    },
    "type": "DOCKER",
    "volumes": []
},
"args": ["hello"],
"cpus": 0.2,
"mem": 32.0,
"instances": 1
}

But I can't found a way to discover a name of my rails app container because marathon assign names with format: "mesos-uuid". Any idea to solve it? or another way to share volume from containers with marathon?

Jequirity

Answer 1

You can add constraints to your app to force them onto specific hosts.

In a simple case, you could do something like this to force an app onto a specific host:

{
 "instances": 1,
 "constraints": [["hostname", "LIKE", "worker-1"]]
}

The next option brings in attributes. You can tag a known number of hosts (let's say 3) with some custom tag.

Then your app definition could look like this:

{
 "instances": 3,
 "constraints": [
   ["hostname", "UNIQUE"],
   ["your-custom-tag", "GROUP_BY"]
 ]
}

You'd add this to the app definition for both of your apps, resulting in one instance of each running on all three slaves you tagged.

See the a reference doc on setting attributes.

Itis

I don't see how this answers the question. Can anyone explain? – Horst

Answer 2

The answer might help others who are looking for it.

Use docker inspect and search for the environment setting in the JSON with key value "MESOS_TASK_ID".

Now you have a matching CONTAINER_ID and MESOS_TASK_ID pair.

Hope this helps.

UPDATE

For determining that in an automated way, first make sure that you docker daemon can be accessed remotely. Note: This may raise security concerns

Edit /etc/default/docker and add DOCKER_OPTS="-H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375" and restart docker - on all your mesos slaves.

Use the following code, to get the MESOS_SLAVE CONTAINER_ID MESOS_TASK_ID mappings

#!/bin/bash
HOSTS=($(curl -s -X GET -H "Accept: text/plain" http://mesosmaster:8080/v2/tasks | tail -n1 | cut -f3-))
hosts=()
for i in "${HOSTS[@]}"
do
hosts+=($(echo $i | awk -F":" '{print $1}'))
done
host=($(printf "%s\n" "${hosts[@]}" | sort -u))
for host in "${host[@]}"
do
INSTANCES=($(docker -H $host:2375 ps -q))
for container_id in "${INSTANCES[@]}"
do
mesos_id=$(docker -H $host:2375 inspect $container_id | grep MESOS_TASK_ID | awk -F '[="]' '{print $3}')
printf "$host\t$container_id\t$mesos_id\n"
done
done

Hope this will help you.

Lainelainey

thks @Shan, but I need a automatic way to discover the CONTAINER_ID or MESOS_TASK_ID. – Jequirity

@kikicarbonell, I'm currently working on it (Python based script), that will query the Marathon REST API on htttp://marathon:8080/v2/tasks and reads the appId and id values (the key id stores the MESOS_TASK_ID). Then run docker inspect command on all existing dockers and match the MESOS_TASK_ID and generated a CONTAINER_ID pair. I will share once done – Lainelainey

@kikicarbonell, For now, you can use the shell script I have added in the above answer. I'm unable to work on the Python script due to work load. And I'm very bad on processing JSON in Python. Sure I will share it here once done. – Lainelainey

Answer 3

My script based on Shan's script. Takes hostname as first argument and a list of marathon deployment ids as others arguments.

I wish this will be helpfull to someone ;)

I am using marathon 0.11.1 and with this version the mesos container name is the value required for the volumes-from parameter.

#!/bin/bash

find_mesos_id () {
  # Parameters
  HOST="$1"

  # All running containers
  CONTAINERS=`docker -H $HOST:2375 ps | sed 1d | awk '{print $1}'`
  CONTAINERS_ARRAY=($CONTAINERS)

  # for each get MARATHON_ID & MESOS_ID
  for i in "${CONTAINERS_ARRAY[@]}"
  do  
     MARATHON_APP_ID=$(docker -H $HOST:2375 inspect $i | grep MARATHON_APP_ID | awk -F '[="]' '{print $3}' | awk -F '\/' '{print $2}')
     MESOS_CONTAINER_NAME=$(docker -H $HOST:2375 inspect $i | grep MESOS_CONTAINER_NAME | awk -F '[="]' '{print $3}')

     # Print MESOS_ID only for desired container
     if [[ $MARATHON_APP_ID = $2 ]]; then
        printf "{ \"key\": \"volumes-from\", \"value\": \"%s\" }" $MESOS_CONTAINER_NAME
     fi   
  done
}   

if [ "$#" -lt 2 ]; then
  echo "USAGE: bash gen-data-volumes.sh <host> [<marathon-id>]"
  printf "This script takes at least two arguments (%d given).\n" $#
  echo "exit 1."
  exit 1;
fi  

if [ "$#" -gt 1 ]; then
  for ((i=2;i<$#+1;i++)) 
    do  
      printf "%s" $(find_mesos_id $1 ${!i})
      if [[ $i -ne $# ]]; then
        printf ", "
      else 
        echo
      fi  
    done
fi

Kalila

Answer 4

You can run something like to get the Container Id

docker ps --filter label=MESOS_TASK_ID=<mesos-task-id> -q

Sheldonshelduck

Answer 5

Get back your real requirement, I don't think it's better using container_id to comunicate with other container(s) than ipaddress. Though you are using zookeeper, you may use it as name service server——when "rails app container" is up, it could register itself in zk at specific node and then other app in other container(s) could find what they want there.

Hypabyssal