ECS tasks desired count not working

Question

I have three EC2 instances, with a classic load balancer. Ideally I should have two tasks running in two instances. So when creating the service I made the desired count of the tasks to 2.

My problem arises when I try to run new version of the task definition. I update the service to run the new task definition. So it should theoretically run two updated tasks replacing the old ones, since i have three ec2 running.

What happens actually is only one updated task is running together with the old tasks. So altogether 3 tasks running even though the desired count is set to 2, as you are able to see in the given image.

Does anyone know a solution for this ?

Hollah

Answer 1

When using a classic load balancer, you can only map static ports on the ec2 instance.

Your deployment settings are: min-health: 100% max-healthy: 200%

The new version of the service would require two more hosts available with the free tcp port you requested. Since you only have 3 servers in the cluster, this condition will not be satisfied. You can either add more servers to your cluster, or use the Application Load Balancer (ALB) which will integrate with docker dynamic port mapping.

Update regarding security groups: To manage security groups, you can tag a security group with another. For example, tag your ALB with 'app-gateway-alb' which allows specific ports from outside your network, then on the container have a security group which allows ANY TCP from 'app-gateway-alb' this is achieved by putting the security group ID in the text box where you would generally put the CIDR rule.

Plummer

Thank you for your information it's very valuable. After this I meddled with the percentages and found that min health 50% and max health 100% works for my requirement. However we will need to migrate to ALB. Through researching i found that even though dynamic port mapping work, the security groups of the EC2s wont be updated according to the random ports. Therefore it poses a problem in using ALB for now. Are you aware of any alternate for this scenario ? – Hollah

A bit late to the party, but the alternative to the mentioned scenario is to keep both the ALB and the instance in the same Security Group. You can safely whitelist all TCP ports in the security group as long as it's only for other resources within the same security group. In short, Security Group whitelists all TCP to itself. – Algol