How can I disable default request headers from apache httpclient 4?

Question

I am using apache common httpclient 4.3.3 to make http 1.0 request. Here is how I make the request

HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
post.setProtocolVersion(new ProtocolVersion("HTTP", 1, 0));

 // trying to remove default headers but it doesn't work
post.removeHeaders("User-Agent");
post.removeHeaders("Accept-Encoding");
post.removeHeaders("Connection");

post.setEntity(new ByteArrayEntity(ba) );

HttpResponse response = client.execute(post);

However, i can see that there are other headers automatically added to my request to the server like

Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.3 (java 1.5)
Accept-Encoding: gzip,deflate

How can I tell httpclient not to include any other headers? I tried to removed those headers with post.removeHeaders(xxxx) but it doesn't work. Can you show me how?

Thanks,

Williamson

Answer 1

If you call HttpClientBuilder.create(), you will have a httpClientBuilder. And httpClientBuilder has a lot config for default headers and this will be used to make intercepters( ex: RequestAcceptEncoding ).

For example, RequestAcceptEncoding, which implements HttpRequestInterceptor, makes Accept-Encoding: gzip,deflate header when HttpProcessor.process() is invoked. And httpProcessor.process() will be invoked just before invoking final CloseableHttpResponse response = this.requestExecutor.execute(route, request, context, execAware);

You can see this code at org.apache.http.impl.execchain.ProtocolExec of httpclient-4.3.6 line 193.

If you want to remove Accept-Encoding: gzip,deflate, call HttpClientBuilder.disableContentCompression() like below.

HttpClient client = HttpClientBuilder.create().disableContentCompression().build();

In short, HttpClientBuilder has a lot of flags to disable/enable HttpRequestInterceptor. If you disable/enable those HttpRequestInterceptor, you can exclude/include default headers.

Sorry for my poor English, and hope you get what I mean.

Sellers

Answer 2

CloseableHttpClient hc = HttpClients.custom()
        .setHttpProcessor(HttpProcessorBuilder.create().build())
        .build();

The code snippet above demonstrates how to create an HttpClient instance with an empty (no-op) protocol processor, which guarantees no request headers will ever be added to outgoing messages executed by such client

Triage

I am not sure what kind of additional explanations are to be expected here. The code snippet above demonstrates how to create an HttpClient instance with an empty (no-op) protocol processor, which guarantees no request headers will ever be added to outgoing messages executed by such client – Triage

this method remove all headers! you must config this – Malposition

Answer 3

You want to do your 'cleanup' at the end, after HttpClient is done modifying the request. You can do this by calling addInterceptorLast on HttpClientBuilder as below.

HttpClient client = HttpClientBuilder.create().addInterceptorLast( 
                        new HttpRequestInterceptor() {
                            public void process(HttpRequest request, HttpContext context){
                                request.removeHeaders("Host");
                                request.removeHeaders("Connection");
                            }
                        }   
                ).build();

I create an anonymous class implementing HttpRequestInterceptor. Take whatever header modifications you need done before the request is sent, and put them in the process method.

Evelynneven

Does this ensure that once the request is sent to the destination it would not have the Host and Connection headers? As far as I know HttpClient by default would add these headers irrespective of us manually removing them or not. Also, when is the process() method called? I tried debugging this but I see no call being made. – Frontality

Answer 4

I think you could add your implementation of HttpRequestInterceptor with client.addRequestInterceptor()
or (better)
remove all interceptors that add headers to the request (RequestUserAgent, RequestDefaultHeaders, RequestClientConnControl, RequestAcceptEncoding, ...).

Removing them is also easy:

client.removeRequestInterceptorByClass(RequestAcceptEncoding.class);

Seraphine

i don't see the method removeRequestInterceptorByClass(xxxx) for HttpClient. Where can I set that method? – Williamson

hc.apache.org/httpcomponents-client-4.3.x/httpclient/apidocs/… But I saw the whole API is deprecated for 4.3 – Seraphine