Status Code 460 on Application Load Balancer

Question

I'm seeing a concerning amount of 460 status codes in the logs of my Application Load Balancer. I don't see any patterns on these codes in regards to times, servers, or request URLs. According to this forum post, the 460 means:

The client has closed the connection with the ALB before the idle timeout has kicked in on either the front-end or the back-end connection.

I can see the request making it to the backend server and the backend processes the request without issue and very quickly. Why are these errors happening? This ALB does a significant amount of traffic with 6-8 backend servers.

Example ALB Log:

https 2017-01-30T22:46:27.451363Z app/LOAD-BALANCER/bbab458ad0b80d X.X.X.X:55999 10.5.X.X:80 0.000 -1 -1 460 - 132 0 "GET https://www.website.com:443/app/page HTTP/1.1" "-" ECDHE-RSA-AES128-SHA TLSv1 arn:aws:elasticloadbalancing:us-west-2:743462462234:targetgroup/TARGET-GROUP/e6120e5adr245b79107e "Root=1-588fc23e-77aea5adf4534af3de09659d13a08"

Example NGINX log from the backend:

X.X.X.X 1485807955.048 www.website.com /app/page - GET 200 - 0.056 24 text/html; charset=UTF-8 -

Dufresne

How did you end up fixing this? The accepted answer talks about altering the client timeout, which is not feasible when the client is a browser... Did you find anything else relevant to the problem? – Orville

Answer 1

The documentation for the status code 460 is updated for Application Load Balancers.

This error occurs when the load balancer received a request from a client, but the client closed the connection with the load balancer before the idle timeout period elapses.

Check whether the client timeout period is greater than the idle timeout period for the load balancer. Ensure that your target provides a response to the client before the client timeout period elapses, or increase the client timeout period to match the load balancer idle timeout, if the client supports this.

You can read the full doc here: http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-troubleshooting.html#http-460-issues

Riba

Is 'client' in this instance a users browser? In what situation would a users browser terminate a connection? – Orville

it could be anything, even a curl request. If it's a browser, you can see this: https://mcmap.net/q/209905/-browser-timeouts – Riba

If the client has closed the connection, who will receive the 460 response? – Garonne

Nobody will receive it. It just shows up in the ALB logs. – Duplet

Answer 2

There is probably a clue in this sequence:

0.000 -1 -1 460 -

The fields are request_processing_time, target_processing_time, response_processing_time, elb_status_code, target_status_code

The target_processing_time and response_processing_time fields being -1 suggest that there was a problem dispatching the request to the target host, as per http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html

Check that your target is getting the request, there could be some configuration or networking problem between the ALB and the target. ALB inserts a tracing header X-Amzn-Trace-Id into the request when it sends it to the target, perhaps log these on your NGINX backend and see if you're getting the specific requests that are failing.

I've been dealing with a similar problem and it appears to be related to our hosts dropping TCP packets due to TCP offload on our windows hosts.

Offoffbroadway