How can I restart the airflow server on Google Composer?

Question

When I need to restart the webserver locally I do:

ps -ef | grep airflow | awk '{print $2}' | xargs kill -9
airflow webserver -p 8080 -D

How can I do this on Google Composer? I don't see an option to restart the server in the console.

Monoceros

Why do you want to restart? I think that this command can cover the function you are looking for. – Homothallic

@RubénC. When I change a plugin, it doesn't seem to take effect until I restart the server. Which of those commands will restart the server? update? – Monoceros

While you'd normally need to restart the webserver to pick up plugins when running Airflow locally, The managed Airflow webserver in Composer automatically loads plugins uploaded to the environment's Cloud Storage bucket. – Hype

Answer 1

Since Cloud Composer is an Apache Airflow managed service, it is not possible to restart the whole service. You can restart though the single instances of the service, as described here, but this will not help to apply to the plugin changes.

To apply the plugin changes, you should install the plugin after loading it into the plugin folder, automatically created when you created the environment:

gcloud composer environments storage plugins import --environment ENVIRONMENT_NAME \
    --location LOCATION \
    --source PATH_TO_LOCAL_FILE \
    --destination PATH_IN_SUBFOLDER

DAG updates are performed automatically and some other must be applied using

gcloud composer environments update

command.

Kodiak

Answer 2

A way to restart Composer server is to add a 'dummy variable' into the 'Environment Variables' of GCP Composer UI. After submitting, it will restart to include this change.

Bebeeru

This worked for Composer 2+ – Overstrain

Answer 3

EDIT: Only works for 1.13.1 >= Composer version < 2.0.0

Restarting the Airflow has recently been introduced as a feature in preview here.

TL;DR the command is:

gcloud beta composer environments restart-web-server ENVIRONMENT_NAME
  --location=LOCATION

Slick

As of Jan. 2022 this does not work for Composer > 2 (gcloud.beta.composer.environments.restart-web-server) FAILED_PRECONDITION: Web server restart feature is not supported for Cloud Composer version 2.0.0 or newer. – Overstrain