AWS MWAA Pricing

Question

It is not clear to me if the aws mwaa documentation says the following "With Amazon Managed Workflows for Apache Airflow (MWAA) you pay only for what you use. There are no minimum fees or upfront commitments. You pay for the time your Airflow Environment runs..", why there is a cost for environment 24/7? If I want to run ETL job only once per day, do I have to pay for the environment 24h?

Ganef

Amazon Managed Workflows for Apache Airflow (MWAA) Pricing – Amazon Web Services -- prices are shown as per hour. – Typify

On that page, first charge is environment - Environment charge Instance usage (in hours) = 31 days x 24 hrs/day = 744 hours x $0.74 (price per hour for a medium environment in the US East (N. Virginia) region) = $ 550.56. It's whole day charge? – Ganef

Hey @JohnRotenstein, I think I maybe did not make myself clear enough. My main concern is, can I use mwaa once per day(let's say 30 minutes) and only pay for that, or I'll pay for compute for those 30 minutes, but also will pay for the environment for the whole day? If I have to pay for the whole day then using Step Functions is a much more cost-effective solution for me. – Ganef

I'm not very familiar with MWAA, but it looks like you would pay for the Environment full-time (since it needs to detect when to activate a job), and you pay additional for workers but only when they are running. If you are just triggering work once per day, then using Airflow is a bit of overkill. – Typify

Answer 1

MWAA can not be paused/stopped in the way that for example ec2 instances can. Even if there are no jobs active or running, the MWAA instance is still running and generating charges.

Doha

Answer 2

why there is a cost for environment 24/7?

MWAA is not a stateless service. It is a service for stateful Airflow deployments, which means it requires continuous compute resources to run.

Amazon Managed Workflows for Apache Airflow Pricing

If I want to run ETL job only once per day, do I have to pay for the environment 24h?

Yes. When your ETL job is not running, Airflow is still deployed and running.

Amelioration

Answer 3

Just to elaborate on the accepted answer, which is correct...

The basic functionality of Apache Airflow provides 2 components: A web server that Data Engineers can interact with to do things like:

• review what job instances were run and their status
• See how the DAGs are organized
• review errors
• execute ad-hoc runs of a DAG

In addition, Apache Airflow has a scheduler that spawns containers using AWS Fargate. It is these containers that execute your ETL jobs

Both of these components of the applications are running 24x7 and you need to pay for the resources required to run these components.

The actual Fargate containers executing your ETL jobs only exist for the duration of each ETL job execution. So, you only pay for the execution time of that ETL job.

Novah