I am using the Amazon ECS "Deploy Task Definition" Action for GitHub Actions template to deploy my app. It sometimes get stuck on this step:
Deploy Amazon ECS task definition
It can get stuck for a long time, between 30 - 60 minutes, before terminating. I am able to login to the AWS console in ECS and see the Task Definition it has created, and then Run a new Task from the newly created Task Definition manually. The result is good as I see the task running succesfully.
When I look at the logs as to why it got stuck in the first place, I have seen error messages such as:
(service myService) was unable to place a task because no container instance met all of its requirements. The closest matching (container-instance id23244214) has insufficient CPU units available.
(service myService) was unable to place a task because no container instance met all of its requirements. The closest matching (container-instance id23244214) has insufficient memory available.
I have 2 vCPUs and 2 (GiB) of memory on my EC2 instance and my task definition does not go over that limit. Since I can just create the task in the ECS dashboard, it doesn't seem like the EC2 instance is lacking memory or VCPU.
Finally, when I see an error message like the two above, I try to redeploy again via GitHub. The GitHub Action passes all its jobs as normal, but it again gets stuck on the Task Definition Step. I then stop receiving any event logs after the last memory/cpu insufficient allocation in Amazon ECS console.
It eventually fails with the error message:
Error: Resource is not in the state servicesStable
Here is my Task
{
"ipcMode": null,
"executionRoleArn": null,
"containerDefinitions": [
{
"dnsSearchDomains": null,
"environmentFiles": null,
"logConfiguration": null,
"entryPoint": null,
"portMappings": [
{
"hostPort": 80,
"protocol": "tcp",
"containerPort": 3000
}
],
"command": null,
"linuxParameters": null,
"cpu": 0,
"environment": [],
"resourceRequirements": null,
"ulimits": null,
"dnsServers": null,
"mountPoints": [],
"workingDirectory": null,
"secrets": null,
"dockerSecurityOptions": null,
"memory": null,
"memoryReservation": null,
"volumesFrom": [],
"stopTimeout": null,
"image": null,
"startTimeout": null,
"firelensConfiguration": null,
"dependsOn": null,
"disableNetworking": null,
"interactive": null,
"healthCheck": null,
"essential": true,
"links": null,
"hostname": null,
"extraHosts": null,
"pseudoTerminal": null,
"user": null,
"readonlyRootFilesystem": null,
"dockerLabels": null,
"systemControls": null,
"privileged": null,
"name": "container"
}
],
"placementConstraints": [],
"memory": "1957",
"taskRoleArn": null,
"family": "family",
"pidMode": null,
"requiresCompatibilities": [
"EC2"
],
"networkMode": null,
"runtimePlatform": null,
"cpu": "2048",
"inferenceAccelerators": null,
"proxyConfiguration": null,
"volumes": []
}
What could be causing this?