How to mount ebs volume(or part) to fargate container

Question

I'm using ECS Fargate to host containers in a service. The service is tied to the target group in order to load balance. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-task-storage.html

This link mentions one can use EFS for persistent storage. IMO EFS is slow. EBS is faster and better. But I can't find a way to share EBS volume directly or indirectly(EBS attached to EC2) with a set on containers so all containers see the same data as well as can write to it and the storage persists even after the containers die

Is there a way to have common storage for all containers on Fargate(I don't mean ECS offering where tasks are hosted on EC2)?

Jilljillana

Answer 1

There are a couple of things to unpack here:

1. Today ECS/Fargate does not support mounting EBS volumes. This support is something we would like to introduce on but it's not available to date.
2. Even though you were able to attach an EBS volume to a task, consider that EBS is block storage and so to be able to share its content with different systems (ECS/Fargate tasks in this case) you'd need to have some sort of clustered file system OR a software/logic that arbitrates access from one system only at a time.

When you say EFS is slow is because you have already tested it (and did not meet your requirements) or is this an assumption?

Cherian

I can't speak to OPs experience, but a yarn install that takes ~6min on my device took ~44min on ECS (4 vCPU x 16GB) and most of the time was spent on linking which, iiuc, means I'm bound by IOPS and latency. node_modules is ~5GB – Negotiation

I imagine there are plenty of use-cases where a workload isn't steady and provisioned throughput doesn't make sense, but where more performance is needed than what the burst balance would provide. It'd be great to be able to go into a "max throughput" mode with improved performance and just pay for the throughput that exceeds my burst balance. I'd get the speed when I need it and wouldn't have to actively manage provisioned throughput up and down. – Negotiation

This answer should be updated based on this answer: https://mcmap.net/q/1905917/-how-to-mount-ebs-volume-or-part-to-fargate-container (EBS volumes are now supported as of 1/2024.) – Karnak

Answer 2

As of Jan 2024, AWS Fargate supports mounting EBS volumes for Linux containers: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ebs-volumes.html

There are some limitations, IMO the biggest one is:

You can attach at most one Amazon EBS volume to each Amazon ECS task, and it must be a new volume. You cannot attach an existing Amazon EBS volume to a task. However, you can configure a new Amazon EBS volume at deployment using the snapshot of an existing volume.

So I'd say it rules out your case of "common storage" for all the containers. It'd be good for low-latency reads on large data though.

Margalo

Answer 3

What you are after is not supported. Solution switch to using EC2 rather than using Fargate.

Docker volumes are only supported when using the EC2 launch type or external instances.

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-volumes.html#docker-volume-considerations

Vanhomrigh