Airflow KubernetesPodOperator - using a ConfigMap value as an environmental variable
Asked Answered
T

0

6

I have a Kubernetes ConfigMap called test that contains a key foobar with some value. I would like to use that key's value in an environmental variable.

import datetime
import os
from airflow import models
from airflow.contrib.kubernetes.volume import Volume
from airflow.contrib.kubernetes.volume_mount import VolumeMount
from airflow.contrib.operators import kubernetes_pod_operator

YESTERDAY = datetime.datetime.now() - datetime.timedelta(days=1)

cm_test = ['test']

volume_mount = VolumeMount('test',
                            mount_path='/config/',
                            sub_path=None,
                            read_only=False)
volume_config = {
    'configMap': {
        'name': 'test'
    }
}
volume = Volume(name='test', configs=volume_config)

with models.DAG(
        dag_id="test_env",
        schedule_interval=datetime.timedelta(days=1),
        start_date=YESTERDAY) as dag:

    
    kubenetes_template_ex = kubernetes_pod_operator.KubernetesPodOperator(
        task_id="test_env",
        name="test_env",
        namespace="foobar",
        image="bash",
        cmds=["printenv"],
        arguments=[],
        volumes=[volume],
        volume_mounts=[volume_mount],
        configmaps=cm_test,
        env_vars={
            'MY_ENV_VAR': '/config/foobar'
        }
    )

What's working: the ConfigMap is available in the mounted volume, so I can do ls /config and foobar is shown.

What's not working: I would like to set an environmental variable with the value from foobar from the mounted configmap. The current code doesn't work, as the literal string value /config/foobar is given to the env var.

Transect answered 23/3, 2021 at 9:13 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.