How to handle HTTP/2 GOAWAY with java.net.HttpClient?

I am trying to continuously send GET and POST requests to a REST API every few minutes. The issue is that after exactly 1000 requests I receive a GOAWAY frame (and an IOException):

The GOAWAY frame (type=0x7) is used to initiate shutdown of a connection or to signal serious error conditions.
_{§ 6.8, RFC 7540}

I did a fair bit of research and found that not only is 1000 requests nginx's default maximum, Cloudfront (related Chromium issue) and Discord also exhibit the same behavior.

I tried to reproduce this problem with a local nginx server with the default HTTP/2 configuration:

server {
    listen 443 http2 ssl;
    http2_max_requests 1000;
    ...
}

var client = HttpClient.newBuilder()
        .version(HttpClient.Version.HTTP_2)
        .build();

for (var i = 0; i < 1100; i++) {
    var url = URI.create(String.format("https://localhost/images/test%d.jpg", i));

    var request = HttpRequest.newBuilder().uri(url).build();

    client.send(request, HttpResponse.BodyHandlers.discarding());
    System.out.printf("Image %d processed%n", i);
}

And after approximately 1000 requests, I get a GOAWAY error as expected:

...
Image 998 processed
Exception in thread "main" java.io.IOException: /127.0.0.1:49259: GOAWAY received

My first thought would be to check if the exception message contains the string "GOAWAY" and then retry the request accordingly:

try {
    client.send(request, HttpResponse.BodyHandlers.discarding());
} catch (IOException e) {
    if (e.getMessage().contains("GOAWAY")) {
        client.send(request, HttpResponse.BodyHandlers.discarding());
    } else throw e;
}

My issue with this approach is that the string comparison seems like it may be fragile. Additionally, since all I have is an IOException with a message, I can't differentiate between GOAWAY frames with a genuine error code (in which case I should probably stop sending requests) and those with NO_ERROR (in which case I could probably retry the request).

How should I correctly deal with/handle GOAWAY errors (apart from using HTTP/1.1 instead)?

A server is entitled to close connections at any time, for any reason.

In the HTTP/2 GOAWAY frame there is the indication of what was the last stream processed by the server, so the client can know what stream needs to be resent when a connection is closed.

Unfortunately, the lastStreamId is not surfaced in java.net.http.HttpClient, so there is no way to know it and take appropriate actions.

Your alternatives could be:

to use other clients that support surfacing the lastStreamId
use a lower level HTTP/2 client where you will have the GOAWAY frame available and therefore access to the lastStreamId

[Disclaimer, I am the Jetty HTTP/2 implementer]
Jetty supports a lower level HTTP/2 client that you can use for your use case - you may want to give it a try. You can find an example of how to use Jetty's HTTP2Client here.

Recommended topics

Hot tags