mwaa Questions

2

Solved

We have an automated upload system for our DAG's to MWAA. Is there a way to have the requirements.txt in the S3 bucket be automatically set to its latest version? I cannot find an option for this i...
Identic asked 17/11, 2021 at 11:48

4

I have an AWS MWAA Airflow v2.0.2 instance running. I would like to have access to the Airflow API. Is this something supported currently? Planned for future releases? There is no mention of this i...
Newmown asked 8/6, 2021 at 9:14

5

We are planning to switch from managing airflow ourselves to Managed Apache Airflow services of AWS. Our original dags use some custom environment variables that need to be set in Managed airflow a...
Bimonthly asked 9/6, 2021 at 7:5

4

Snowflake is not showing in the connections dropdown. I am using MWAA 2.0 and the providers are already in the requirements.txt MWAA uses python 3.7 dont know if this can be a thing Requirements.tx...
A asked 1/6, 2021 at 11:34

5

Solved

I've uploaded my requirementst.txt to a dag and referenced it when creating my Airflow Environment. Here is what my requirements.txt looks like: apache-airflow==1.10.12 oauth2client==4.1.3 google-a...
Dozier asked 4/5, 2021 at 15:42

2

Solved

Does anyone use MWAA in production? We currently have around 500 DAGs running and we see an unexpected behavior with tasks staying in a "queued" state for unknown reasons. Task is in the...
Bellini asked 6/1, 2022 at 12:29

3

Solved

We use Amazon MWAA Airflow, rarely some task as marked as "FAILED" but there is no logs at all. As if the container had been shut down without noticing us. I have found this link: https:/...
Caterinacatering asked 30/4, 2021 at 10:12

2

Solved

I am using AWS's MWAA service (2.2.2) to run a variety of DAGs, most of which are implemented with standard PythonOperator types. I bundle the DAGs into an S3 bucket alongside any shared requiremen...
Daemon asked 13/4, 2022 at 17:44

3

Solved

It is not clear to me if the aws mwaa documentation says the following "With Amazon Managed Workflows for Apache Airflow (MWAA) you pay only for what you use. There are no minimum fees or upfr...
Ganef asked 11/12, 2021 at 20:46

1

I'm currently using AWS Apache Airflow service (MWAA) and I've been needing to run some web scraping code using selenium, I did manage to add "chromedriver" and "selenium" to th...
Viveca asked 16/7, 2021 at 18:14

4

Solved

I am using an AWS Managed Airflow to orchestrate a data pipeline. The problem that I am facing now is that logs are not being printed in the airflow log not on cloud watch. I am getting the followi...
Denison asked 26/8, 2021 at 8:58

0

I have a Docker file which creates an image and then I run it using docker compose together with a container built using a Postgres image. (To set up a local environment of Airflow - we use the mwa...
Oleoresin asked 20/5, 2022 at 7:19

4

hope you are doing well. I wanted to check if anyone has get up and running with dbt in aws mwaa airflow. I have tried without success this one and this python packages but fails for some reason or...
Townsley asked 4/6, 2021 at 13:56

2

I need to use sshoperator in a DAG on AWS Airflow (Amazon MWAA), so I imported the following library in my DAG file from airflow.contrib.operators.ssh_operator import SSHOperator It seems sshopera...
Godman asked 16/6, 2021 at 2:19

1

I have updated some Airflow configuration options in my MWAA environment and it has been in "Updating" state for the last three hours! Does anybody know how to force-delete that environme...
Cockshy asked 6/8, 2021 at 19:16

2

I'm testing the waters for running Apache Airflow on AWS through the Managed Workflows for Apache Airflow (MWAA). The version of Airflow that AWS have deployed and are managing for me is 1.10.12. W...
Lierne asked 24/2, 2021 at 5:4

1

How can I configure airflow (mwaa) so that it will fire at the same time (6am PST) every day regards of when the dag is deployed? I have tried what makes sense to me: set the schedule_interval to ...
Canicular asked 20/2, 2021 at 3:5
1

© 2022 - 2024 — McMap. All rights reserved.