Airflow: set a default value in code when Variable doesn't exist without an exception

Question

I have a little problem, I want to do the typical conditional like

setting_x  = Variable.get('setting_x')
variable = setting_x if setting_x else 0

But since the Airflow model throws an exception when the key doesn't exist is impossible to do it without trycatching and that's not very cool.

Is there any solution that I'm missing to solve that case? I've searched in the whole internet of course, but without a solution yet.

Thanks, Angel

Tyburn

Answer 1

You can set the default for the Variable if it doesn't exist when you're retrieving it with the get method.

variable = Variable.get('setting_x', default_var=0)

https://github.com/apache/airflow/blob/master/airflow/models/variable.py#L127

Riel

Hey! thank you! it works so I will mark it as solved, but I'm having more or less the same problem since the value should be always != None. Let's say another example: – Tyburn

slack_channel = Variable.get('SLACK_CHANNEL', default_var=None) slack_channel = slack_channel if slack_channel else os.environ['SLACK_CHANNEL'] – Tyburn

In that case, it still throwing an exception, that's a little constraint since we can't manage a possible None value without trycatching, don't you think so? Python is not my main language so probably I'm missing something but is strange to me can't manage a possible None value in a model – Tyburn

slack_channel = slack_channel if slack_channel else os.environ['SLACK_CHANNEL'] should work unless your SLACK_CHANNEL environment variable is also None. If you can never have it be None and there is a possibility that the env SLACK_CHANNEL is also None, I would add a check at the end to ensure it is something else -- if not slack_channel: slack_channel = 'DOES NOT EXIST' – Riel

Any way to do this with a Jinja template? {{ var.value.setting_x or 0 }} will error out if the variable setting_x doesn't exist. – Danielldaniella

Answer 2

Shorter version of the previous answer (keyword default_var omitted):

variable  = Variable.get('setting_x', 0)

Psychopathy