Login/Register
Java 9 HttpClient exception when using certain characters in URL query parameters
Asked Answered
Solved javajava-9url-encodinghttp2java-http-client
S

1

13

Here is my sample code. The query is encoded to UTF-8:

HttpRequest request = HttpRequest.newBuilder()
    .header("content-type", "application/json;charset=UTF-8")
    .uri(URI.create("http://localhost:8080/test?param1=test%C5%84"))
    .GET()
    .build();

HttpClient.newBuilder()
    .version(HttpClient.Version.HTTP_2)
    .build()
    .send(request, HttpResponse.BodyHandler.asString(Charset.forName("UTF-8")));

After I run this example I get the following exception:

java.lang.IllegalArgumentException: char=324
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Huffman.codeOf(Huffman.java:559)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Huffman.lengthOf(Huffman.java:524)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.StringWriter.configure(StringWriter.java:79)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.StringWriter.configure(StringWriter.java:62)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.IndexNameValueWriter.value(IndexNameValueWriter.java:64)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.literal(Encoder.java:422)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.header(Encoder.java:245)
at jdk.incubator.httpclient/jdk.incubator.http.internal.hpack.Encoder.header(Encoder.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeadersImpl(Http2Connection.java:927)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeaders(Http2Connection.java:878)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.encodeHeaders(Http2Connection.java:951)
at jdk.incubator.httpclient/jdk.incubator.http.Http2Connection.sendFrame(Http2Connection.java:984)
at jdk.incubator.httpclient/jdk.incubator.http.Stream.sendHeadersAsync(Stream.java:547)
at jdk.incubator.httpclient/jdk.incubator.http.Exchange.lambda$responseAsyncImpl0$8(Exchange.java:322)
at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate.setALPN(SSLFlowDelegate.java:164)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate.access$200(SSLFlowDelegate.java:81)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:340)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:215)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$TryEndDeferredCompleter.complete(SequentialScheduler.java:315)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SSLFlowDelegate$Reader.incoming(SSLFlowDelegate.java:242)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.incomingCaller(SubscriberWrapper.java:388)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.onNext(SubscriberWrapper.java:343)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SubscriberWrapper.onNext(SubscriberWrapper.java:58)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:739)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)

char=324 means decoded ń from query

When I read stack trace I found jdk.incubator.http.Stream<T> this method in the class:

private void setPseudoHeaderFields() {
    HttpHeadersImpl hdrs = requestPseudoHeaders;
    String method = request.method();
    hdrs.setHeader(":method", method);
    URI uri = request.uri();
    hdrs.setHeader(":scheme", uri.getScheme());
    // TODO: userinfo deprecated. Needs to be removed
    hdrs.setHeader(":authority", uri.getAuthority());
    // TODO: ensure header names beginning with : not in user headers
    String query = uri.getQuery();
    String path = uri.getPath();
    if (path == null || path.isEmpty()) {
        if (method.equalsIgnoreCase("OPTIONS")) {
            path = "*";
        } else {
            path = "/";
        }
    }
    if (query != null) {
        path += "?" + query;
    }
    hdrs.setHeader(":path", path);
}

In this method uri.getQuery() is used which gives us the decoded query and causes the above exception.

When I used uri.getRawQuery() in debug mode (which gave us the encoded query) everything went fine.

My question is: is that a bug or intentional usage? If it is not a bug, how do I avoid the exception?

Submission answered 14/6, 2018 at 10:57 Comment(3)
test%C5%84 maybe (extra %)? As in "...?param=" + URLEncoder.encode("ń", "UTF-8") – Hardnett
Sorry, it is my mistake. There should be test%C5%84, but in method setPseudoHeaderFields() I get ...testń, and future exception. – Submission
It seems that the Huffman compression uses a simplistic char to byte conversion à la ISO-8859-1. So to my very limited experience it seems a bug. You might revert to Base64: "...?param1=" + Base64.getEncoder().encode("ń") but then you need to decode the parameter yourself too. – Hardnett
B
13

It is a bug:

java.lang.IllegalArgumentException with jdk.incubator.httpclient when using some UTF-8 characters

Some UTF-8 characters like "š" can't be used with HttpClient. However others like "å" can be.

See: https://bugs.openjdk.java.net/browse/JDK-8201238

Java 9/10 HttpClient is incubated. It's not production-ready. It states so very clearly in the package name. As such, it's quite possible that it may contain bugs.

It will be released properly in Java 11 (currently scheduled for release 25th September 2018) as part of JEP 321.

Bottle answered 14/6, 2018 at 11:57 Comment(3)
Note: the HTTP Client API will be standardized in Java 11 and will be moved to a new package (and module) named java.net.http. (JEP 321) – Bunns
@Bunns I am so happy for this, finally a standard http client in the jdk itself! – Angry
You can plan to migrate to Java-11 for using the standardized http module.. I can confirm that the code in question works appropriately with JDK/11 (jdk-11-ea+17). – Firenew

© 2022 - 2024 — McMap. All rights reserved.