Restart a single exited container in an ECS task

Question

I have a container that is part of an ECS task definition, which I have marked as essential=false, because if this container goes down, I do not want the ECS agent to take down the other containers in the task. Making the container "non-essential" has achieved the desired result in my case: that container crashes, and the other containers on the task do not get taken down or restarted.

However, I do want this non-essential container to be independently restarted. Is there any built-in way to accomplish this? Basically, if the container exits, run docker start or docker restart on that container (which we are currently having to do manually). I have not had any luck so far with the documentation or from exploring the AWS console.

Exit

Are you / Have you tried running the container as part of an ECS service? If this container can crash and not have an impact on the others in your task, can you run it as a separate task? – Itu

have you tried other container image with hypervisord ? Hypervisord allows you to restart, container process if it's crashed and ensures container is running.. So, with hypervisord container will not exit.. your app will be restarted by hpervisord if it crashes.. – Wagon

@Itu good idea! This would require some changes that our team won't be able to make at the moment, but this sounds like a really promising option. I will update here if we implement this. Thanks! – Exit

@AnkushTehale interesting idea! I will look into this. Thanks! – Exit

Did the answer below helped you? Thanks! – Treasury

@Exit did you solve your issue by converting to Services? I am also experiencing this issue, and my reason for having multiple Containers under one task definition is to communicate between the containers within the Cluster. – Fictionalize

@Fictionalize sorry, I haven't made any progress on this. Priority has shift at my work, and this has been put on hold :( – Exit

Answer 1

Docker provides a restart policy that would be useful in your case (--restart always), however, based on this thread, ECS does not support restarting existing containers.

The suggested and accepted workaround was:

ECS supports this use-case through the concept of a "service". Services work to continuously make the reality (known state) match the desired state, including the desired number of running tasks you specify. If a task started by a service stops, the service will create a new task to replace it. Services help you manage the number of copies you want running, deployments, binding to and unbinding from load balancers, respond to load balancer health checks, and integrate with auto scaling so your service can scale in or out automatically. You can check out the documentation for more detail.

Treasury

I will look into this. I know we do some configuration with user data already. Do you have any more specific suggestions as to how user data could be used to restart an exited container? – Exit

@elethan, I was wondering if you could set Docker restart policy through user-data: --restart always. However, it doesn't seems to be supported by ECS, as discussed in this thread. I'll update the answer with the findings. – Treasury

Thanks Fabio! I won't be able to test this solution for a little while because: crunch time at work. But this info is helpful so I gave you a +1! I'll update once/if I try this out! – Exit

Starting today, restart policy is supported: Amazon ECS provides the ability to restart containers without requiring a task relaunch – Cathode