Why does AWS CodeDeploy AllowTraffic take 3 minutes per instance?

Question

I recently started using AWS CodeDeploy and noticed that the AllowTraffic step consistently takes between 3 and 4 minutes per instance. I've configured the health check interval to be 10 seconds and the health threshold to be 2, so I expect it to take 20 seconds. I'm using a Network Load Balancer.

I have polled on the NLB's deployment group using describe-target-health and confirmed that the target is in the initial state for the 3+ minutes that CodeDeploy is waiting. I have also confirmed that the server on the health check port is responsive at the very beginning of the three minutes.

What are other possible reasons for CodeDeploy / NLB to be so slow?

Unionize

Answer 1

The extra time you are witnessing is not due to health check but because of initial registration of a target to the NLB takes time.

When you register a new target to your Network Load Balancer, it is expected to take between 30 and 90 seconds (can go upto 120 seconds) to complete the registration process. After registration is complete, the Network Load Balancer health check systems will begin to send health checks to the target. A newly registered target must pass health checks for the configured interval to enter service and receive traffic.

For example, if you configure your health check for a 10 second interval, and require 2 health checks to become healthy, so the the minimum time an instance will start receiving traffic is [30sec -to- 120sec] (registration) + [20sec] (health check)

An ALB is not affected by this initial registration delay so it registers instances much faster. It is just how the NLB operates at this point in time.

Malloch

Thanks! Is this documented somewhere publicly? – Unionize

Also I'm seeing times that are equivalent to 30% longer on top of the health check time. – Unionize

I couldnt find a direct reference to this extra registration delay ... but this is the next best thing: docs.aws.amazon.com/elasticloadbalancing/latest/network/… It mentions: "We recommend that you specify a value of at least 120 seconds to ensure that requests are completed." – Malloch