Java HTTP Client Request with defined timeout

Question

I would like to make BIT (Built in tests) to a number of server in my cloud. I need the request to fail on large timeout.

How should I do this with java?

Trying something like the below does not seem to work.

public class TestNodeAliveness {
 public static NodeStatus nodeBIT(String elasticIP) throws ClientProtocolException, IOException {
  HttpClient client = new DefaultHttpClient();
  client.getParams().setIntParameter("http.connection.timeout", 1);

  HttpUriRequest request = new HttpGet("http://192.168.20.43");
  HttpResponse response = client.execute(request);

  System.out.println(response.toString());
  return null;
 }


 public static void main(String[] args) throws ClientProtocolException, IOException {
  nodeBIT("");
 }
}

-- EDIT: Clarify what library is being used --

I'm using httpclient from apache, here is the relevant pom.xml section

 <dependency>
   <groupId>org.apache.httpcomponents</groupId>
   <artifactId>httpclient</artifactId>
   <version>4.0.1</version>
   <type>jar</type>
 </dependency>

Squalid

What library are you using for all of the HTTP functionality? – Nutt

Answer 1

import org.apache.http.client.HttpClient;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpParams;

...

    // set the connection timeout value to 30 seconds (30000 milliseconds)
    final HttpParams httpParams = new BasicHttpParams();
    HttpConnectionParams.setConnectionTimeout(httpParams, 30000);
    client = new DefaultHttpClient(httpParams);

Reprography

What does this do? A timeout of what does it set? Please see #3001267 – Squalid

Apache's HttpClient has two separate timeouts: a timeout for how long to wait to establish a TCP connection, and a separate timeout for how long to wait for a subsequent byte of data. HttpConnectionParams.setConnectionTimeout() is the former, HttpConnectionParams.setSoTimeout() is the latter. – Cellarage

What is the default timeout? Is it 0=infinite? – Harness

http.connection.timeout Integer The timeout until a connection is established. A value of zero means the timeout is not used. – Locker

To future readers: It should be noted that the classes referenced here are now deprecated (as of 2016-07-05), though this was probably a good answer when it was posted in 2010. See this answer: https://mcmap.net/q/204474/-java-http-client-request-with-defined-timeout for something a bit more current. – Diwan

Answer 2

If you are using Http Client version 4.3 and above you should be using this:

RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(30 * 1000).build();
HttpClient httpClient = HttpClientBuilder.create().setDefaultRequestConfig(requestConfig).build();

Forest

Thanks! I spent 20 minutes at least trying to figure out how to get rid of the deprecated commands to do this before finding your answer. – Olivette

Forgot SocketTimeout. Also, does "default" mean it's default for all HttpClient that the HttpClientBuilder from create() will give through build() method? Or just the ones that pass through setDefaultRequestConfig(requestConfig)? Am I making sense? – Dorladorlisa

@Dorladorlisa Your question is a bit confusing :). The default request config will be used for all HttpPost, HttpGet,... instances using 'only' that HttpClient. If you made another HttpClient instance then this request config will not be used with that client. – Forest

Answer 3

HttpParams is deprecated in the new Apache HTTPClient library. Using the code provided by Laz leads to deprecation warnings.

I suggest to use RequestConfig instead on your HttpGet or HttpPost instance:

final RequestConfig params = RequestConfig.custom().setConnectTimeout(3000).setSocketTimeout(3000).build();
httpPost.setConfig(params);

Zelazny

"using the code above" << SO is not a forum and answers move up/down due to several factors, so we don't know which code you're referring to. Could you say something like "using the code from xyz's answer"? – Dorladorlisa

Btw, neat answer. But when compared to Thunder's answer, what's the difference between setting the RequestConfig into every HttpPost as opposed to setting it as default into the HttpClient? – Dorladorlisa

Yes, you are right, the code Above refers to the "accepted answer" which shouldn't change in time. I will update my answer anyway.. – Zelazny

Answer 4

It looks like you are using the HttpClient API, which I know nothing about, but you could write something similar to this using core Java.

try {

   HttpURLConnection con = (HttpURLConnection) new URL(url).openConnection();
   con.setRequestMethod("HEAD");
   con.setConnectTimeout(5000); //set timeout to 5 seconds
   return (con.getResponseCode() == HttpURLConnection.HTTP_OK);

} catch (java.net.SocketTimeoutException e) {
   return false;
} catch (java.io.IOException e) {
   return false;
}

Emmons

This will not affect the timeout for Apache HTTP Client. – Reprography

HttpURLConnection's notion of timeout is insufficient. If the server begins to respond, but then hangs, the timeout is never reached. Apache's HttpClient is a better choice because of this difference. – Cellarage

Answer 5

The said method with highest up's by Laz is deprecated from version 4.3 onwards. Hence it would be better to user the Request Config Object and then build the HTTP Client

    private CloseableHttpClient createHttpClient()
        {
        CloseableHttpClient httpClient;
        CommonHelperFunctions helperFunctions = new CommonHelperFunctions();
        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
        cm.setMaxTotal(306);
        cm.setDefaultMaxPerRoute(108);
        RequestConfig requestConfig = RequestConfig.custom()
            .setConnectTimeout(15000)
            .setSocketTimeout(15000).build();
        httpClient = HttpClients.custom()
            .setConnectionManager(cm)
            .setDefaultRequestConfig(requestConfig).build();
        return httpClient;
        }

The PoolingHttpClientConnectionManager is user to set the max default number of connections and the max number of conncetions per route. I have set it as 306 and 108 respectively. The default values will not be sufficient for most of the cases.

For setting Timeout: I have used the RequestConfig object. You can also set the property Connection Request Timeout for setting timeout for waiting for connection from Connection manager.

Abyss

Answer 6

I found that setting the time out settings in HttpConnectionParams and HttpConnectionManager did not solve our case. We're limited to using org.apache.commons.httpclient version 3.0.1.

I ended up using an java.util.concurrent.ExecutorService to monitor the HttpClient.executeMethod() call.

Here's a small, self-contained example

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.EntityEnclosingMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.*;

/**
 * @author Jeff Kirby
 * @since <pre>Jun 17, 2011</pre>
 */
public class Example {

   private static final String SITE = "http://some.website.com/upload";
   private static final int TIME_OUT_SECS = 5;

   // upload a file and return the response as a string
   public String post(File file) throws IOException, InterruptedException {
      final Part[] multiPart = { new FilePart("file", file.getName(), file) };
      final EntityEnclosingMethod post = new PostMethod(SITE);
      post.setRequestEntity(new MultipartRequestEntity(multiPart, post.getParams()));
      final ExecutorService executor = Executors.newSingleThreadExecutor();
      final List<Future<Integer>> futures = executor.invokeAll(Arrays.asList(new KillableHttpClient(post)), TIME_OUT_SECS, TimeUnit.SECONDS);
      executor.shutdown();
      if(futures.get(0).isCancelled()) {
         throw new IOException(SITE + " has timed out. It has taken more than " + TIME_OUT_SECS + " seconds to respond");
      }
      return post.getResponseBodyAsString();
   }

   private static class KillableHttpClient implements Callable<Integer> {

      private final EntityEnclosingMethod post;

      private KillableHttpClient(EntityEnclosingMethod post) {
         this.post = post;
      }

      public Integer call() throws Exception {
         return new HttpClient().executeMethod(post);
      }
   }
}

Metzgar

Answer 7

This was already mentioned in a comment by benvoliot above. But, I think it's worth a top-level post because it sure had me scratching my head. I'm posting this in case it helps someone else out.

I wrote a simple test client and the CoreConnectionPNames.CONNECTION_TIMEOUT timeout works perfectly in that case. The request gets canceled if the server doesn't respond.

Inside the server code I was actually trying to test however, the identical code never times out.

Changing it to time out on the socket connection activity (CoreConnectionPNames.SO_TIMEOUT) rather than the HTTP connection (CoreConnectionPNames.CONNECTION_TIMEOUT) fixed the problem for me.

Also, read the Apache docs carefully: http://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/params/CoreConnectionPNames.html#CONNECTION_TIMEOUT

Note the bit that says

Please note this parameter can only be applied to connections that are bound to a particular local address.

I hope that saves someone else all the head scratching I went through. That will teach me not to read the documentation thoroughly!

Jotter

Answer 8

Op later stated they were using Apache Commons HttpClient 3.0.1

 HttpClient client = new HttpClient();
        client.getHttpConnectionManager().getParams().setConnectionTimeout(5000);
        client.getHttpConnectionManager().getParams().setSoTimeout(5000);

Muffler

Answer 9

HttpConnectionParams.setSoTimeout(params, 10*60*1000);// for 10 mins i have set the timeout

You can as well define your required time out.

Adonic