AWS - Batch vs Fargate

Question

I have a docker image. I would like to create a container periodically and execute as a job, say every 1 hour, by creating CloudWatch Rule.

As we are using AWS cloud, I am looking at the AWS Batch service. Interestingly there is also a ECS Scheduled task.

What is the difference between these 2?

Note: I have an init container - that is I have 2 docker containers to run one after another. It seems to be possible with ECS Scheduled Task. But not with Batch.

Tetryl

Answer 1

AWS Batch is for batch jobs, such as processing numerous images or videos in parallel (one container per image/video). This is mostly useful in batch-type workloads for research purposes.

AWS Batch is based on ECS (also supports EC2), and it allows you to simply to run your containers. It does not have specific use-case, it is more generic. If you don't have batch-type projects, then ECS would be probably better choice for you.

Virus

Answer 2

The other answers are spot on. I just wanted to add that we (AWS container team) ran a session at re:Invent last year that covered these options and provided hints about when using one over the other. The session covers the relationship between ECS, EC2 and Fargate (something that is often missed) as well as when to use "raw" ECS, Vs Step Functions Vs Batch as an entry point for running your batch jobs. This is the link to the session.

Nonmetallic

Answer 3

If you want to run two containers in sequence, using AWS Fargate, then you probably want to orchestrate it with AWS Step Functions. Step Functions will allow you to call arbitrary tasks in serial, and it has direct integration with AWS Fargate.

Amazon EventBridge Rule (hourly) ----- uses AWS IAM role to gain permission to trigger Step Functions

|
| triggers
|

AWS Step Functions ----- Uses AWS IAM role to gain permission to trigger Fargate

|
| triggers
|

AWS Fargate (Amazon ECS) Task Definition

AWS Batch is designed for data processing tasks that need to scale out across many nodes. If your use case is simply to spin up a couple of containers in sequence, then AWS Batch will be overkill.

CloudWatch Event Rules

FYI CloudWatch Event Rules still work, but the service has been rebranded as Amazon EventBridge. I'd recommend using the Amazon EventBridge console and APIs instead of Amazon CloudWatch Events APIs going forward.

Doggy