How to get Task ID from within ECS container?

Question

Hello I am interested in retrieving the Task ID from within inside a running container which lives inside of a EC2 host machine.

AWS ECS documentation states there is an environment variable ECS_CONTAINER_METADATA_FILE with the location of this data but will only be set/available if ECS_ENABLE_CONTAINER_METADATA variable is set to true upon cluster/EC2 instance creation. I don't see where this can be done in the aws console.

Also, the docs state that this can be done by setting this to true inside the host machine but would require to restart the docker agent.

Is there any other way to do this without having to go inside the EC2 to set this and restart the docker agent?

Knobby

Answer 1

The technique I'd use is to set the environment variable in the container definition.

If you're managing your tasks via Cloudformation, the relevant yaml looks like so:

  Taskdef:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ...
      ContainerDefinitions:
        - Name: some-name
          ...
          Environment:
            - Name: AWS_DEFAULT_REGION
              Value: !Ref AWS::Region
            - Name: ECS_ENABLE_CONTAINER_METADATA
              Value: 'true'

This technique helps you keep everything straightforward and reproducible.

If you need metadata programmatically and don't have access to the metadata file, you can query the agent's metadata endpoint:

curl http://localhost:51678/v1/metadata

Note that if you're getting this information as a running task, you may not be able to connect to the loopback device, but you can connect to the EC2 instance's own IP address.

Edan

Answer 2

This doesn't work for newer Amazon ECS container versions anymore, and in fact it's now much simpler and also enabled by default. Please refer to this docu, but here's a TL;DR:

If you're using Amazon ECS container agent version 1.39.0 and higher, you can just do this inside the docker container:

curl -s "$ECS_CONTAINER_METADATA_URI_V4/task" \
  | jq -r ".TaskARN" \
  | cut -d "/" -f 3

Here's a list of container agent releases, but if you're using :latest – you're definitely fine.

Haply

The ECS_CONTAINER_METADATA_URI_V4 environment variable is no longer available. You can use ECS_CONTAINER_METADATA_URI instead. – Levileviable

Hmm, not sure: the doc that I linked still mentions _V4 variable. – Haply

Answer 3

Previous answers are correct, here is another way of doing this:

From the ec2 instance where container is running, run this command

curl http://localhost:51678/v1/tasks | python -mjson.tool |less

Covetous

Answer 4

We set it with the so called user data, which are executed at the start of the machine. There are multiple ways to set it, for example: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-console

It could look like this:

#!/bin/bash

cat <<'EOF' >> /etc/ecs/ecs.config
ECS_CLUSTER=ecs-staging
ECS_ENABLE_CONTAINER_METADATA=true
EOF

Important: Adjust the ECS_CLUSTER above to match your cluster name, otherwise the instance will not connect to that cluster.

Detective

Answer 5

From the AWS ECS cli Documentation

Command:

aws ecs list-tasks --cluster default

Output:

{
    "taskArns": [
        "arn:aws:ecs:us-east-1:<aws_account_id>:task/0cc43cdb-3bee-4407-9c26-c0e6ea5bee84",
        "arn:aws:ecs:us-east-1:<aws_account_id>:task/6b809ef6-c67e-4467-921f-ee261c15a0a1"
    ]
}

To list the tasks on a particular container instance

This example command lists the tasks of a specified container instance, using the container instance UUID as a filter.

Command:

aws ecs list-tasks --cluster default --container-instance f6bbb147-5370-4ace-8c73-c7181ded911f

Output:

{
    "taskArns": [
        "arn:aws:ecs:us-east-1:<aws_account_id>:task/0cc43cdb-3bee-4407-9c26-c0e6ea5bee84"
    ]
}

Clergyman

Answer 6

After dumping ALL the environmental variable within my container/ECS Fargate execution environment, with:

    foreach (string key in Environment.GetEnvironmentVariables().Keys)
    {
        string value = Environment.(key);
        Console.WriteLine($"{key} = {value}");
    }

I found that the Environment Variable ECS_CONTAINER_METADATA_URI contains the TaskId/Container runtime ID, with the metadata endpoint and an extraneous value which can be trimmed out.

ECS_CONTAINER_METADATA_URI = http://169.254.170.2/v3/93e1ab7d263746bcbfe4cd94808c7044-3151447892

var taskId  = 
    Environment.GetEnvironmentVariable("ECS_CONTAINER_METADATA_URI_V4")
        .Split("/").Last()
        .Split("-").First();

Bascomb

Answer 7

My ECS solution as bash and Python snippets. Logging calls are able to print for debug by piping to sys.stderr while print() is used to pass the value back to a shell script

    #!/bin/bash
    TASK_ID=$(python3.8 get_ecs_task_id.py)
    echo "TASK_ID: ${TASK_ID}"

Python script - get_ecs_task_id.py

    import json
    import logging
    import os
    import sys
    
    import requests

    # logging configuration
    # file_handler = logging.FileHandler(filename='tmp.log')
    # redirecting to stderr so I can pass back extracted task id in STDOUT
    stdout_handler = logging.StreamHandler(stream=sys.stderr)
    # handlers = [file_handler, stdout_handler]
    handlers = [stdout_handler]
    
    logging.basicConfig(
        level=logging.INFO,
        format="[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s",
        handlers=handlers,
        datefmt="%Y-%m-%d %H:%M:%S",
    )
    
    logger = logging.getLogger(__name__)
    
    
    def get_ecs_task_id(host):
        path = "/task"
        url = host + path
        headers = {"Content-Type": "application/json"}
        r = requests.get(url, headers=headers)
        logger.debug(f"r: {r}")
        d_r = json.loads(r.text)
        logger.debug(d_r)
        ecs_task_arn = d_r["TaskARN"]
        ecs_task_id = ecs_task_arn.split("/")[2]
        return ecs_task_id
        
    
    def main():
        logger.debug("Extracting task ID from $ECS_CONTAINER_METADATA_URI_V4")
        logger.debug("Inside get_ecs_task_id.py, redirecting logs to stderr")
        logger.debug("so that I can pass the task id back in STDOUT")
    
        host = os.environ["ECS_CONTAINER_METADATA_URI_V4"]
        ecs_task_id = get_ecs_task_id(host)
        # This print statement passes the string back to the bash wrapper, don't remove
        logger.debug(ecs_task_id)
        print(ecs_task_id)
    
    
    if __name__ == "__main__":
        main()

Situated

Answer 8

This will do the trick:

aws ecs list-tasks --cluster $CLUSTER_NAME --service $SERVICE_NAME --query "taskArns" --output text

Bash script will make you easier as you don't have to define the defined variables everytime.

#!/bin/bash

export AWS_PROFILE=$AWS_PROFILE
CLUSTER_NAME=$CLUSTER_NAME
SERVICE_NAME=$SERVICE_NAME
CONTAINER_NAME=$CONTAINER_NAME
TASK_ID=$(aws ecs list-tasks --cluster $CLUSTER_NAME --service $SERVICE_NAME --query "taskArns" --output text)

aws ecs execute-command --cluster $CLUSTER_NAME \
    --task $TASK_ID \
    --container $CONTAINER_NAME \
    --interactive \
    --command "/bin/sh"

Exhort

Answer 9

It seems the task ID can be found on the URI path of some of the environment variables available inside the container like the ECS_CONTAINER_METADATA_URI_V4 itself.

Using ECS Exec I've run the env command (lists environment variables) on my task container. I'm using Fargate:

aws ecs execute-command \
  --region us-east-1 \
  --cluster cluster-name \
  --task f630d9ab670d45dc9363b93061aeb5eb \
  --container container-name \
  --interactive \
  --command "env"

Output:

...
ECS_AGENT_URI=http://169.254.170.2/api/f630d9ab670d45dc9363b93061aeb5eb-1708180144
ECS_CONTAINER_METADATA_URI_V4=http://169.254.170.2/v4/f630d9ab670d45dc9363b93061aeb5eb-1708180144
ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/f630d9ab670d45dc9363b93061aeb5eb-1708180144
...

In this case, my task id was f630d9ab670d45dc9363b93061aeb5eb.

So if you don't want to make a request just to get the task ID, you can process the value in these environment variables directly. (using regex, substring, awk, etc)

An example using Java:

String ecsMetadataUrlV4 = "<env var value here>"
List<String> parts = Arrays.asList(ecsMetadataUrlV4.split("/v4/"));
String taskId = parts.get(1).split("-")[0];

I couldn't find documentation for the value itself of these environment variables and structure of these URIs so I'd consider this a hacky solution which may break in the future.

Nawrocki