How can I reuse a HttpClient connection efficiently?

Question

I am doing HTTP POSTs very frequently (>= 1/sec) to an API endpoint and I want to make sure I'm doing it efficiently. My goal is to succeed or fail as soon as possible, especially since I have separate code to retry failed POSTs. There is a nice page of HttpClient performance tips, but I'm not sure if exhaustively implementing them all will have real benefits. Here is my code right now:

public class Poster {
  private String url;
  // re-use our request
  private HttpClient client;
  // re-use our method
  private PostMethod method;

  public Poster(String url) {
    this.url = url;

    // Set up the request for reuse.
    HttpClientParams clientParams = new HttpClientParams();
    clientParams.setSoTimeout(1000);  // 1 second timeout.
    this.client = new HttpClient(clientParams);
    // don't check for stale connections, since we want to be as fast as possible?
    // this.client.getParams().setParameter("http.connection.stalecheck", false);

    this.method = new PostMethod(this.url);
    // custom RetryHandler to prevent retry attempts
    HttpMethodRetryHandler myretryhandler = new HttpMethodRetryHandler() {
      public boolean retryMethod(final HttpMethod method, final IOException exception, int executionCount) {
        // For now, never retry
        return false;
      }
    };

    this.method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, myretryhandler);
  }

  protected boolean sendData(SensorData data) {
    NameValuePair[] payload = {
      // ...
    };
    method.setRequestBody(payload);

    // Execute it and get the results.
    try {
      // Execute the POST method.
      client.executeMethod(method);
    } catch (IOException e) {
      // unable to POST, deal with consequences here
      method.releaseConnection();
      return false;
    }

    // don't release so that it can be reused?
    method.releaseConnection();

    return method.getStatusCode() == HttpStatus.SC_OK;
  }
}

Would it make sense to disable the check for stale connections? Should I be looking at using the MultiThreadedConnectionManager? Of course, actual benchmarking would help but I wanted to check if my code is on the right track first.

Cragsman

It's ironic that I earned the Popular Question badge (1000+ views) despite there being no answers. If you have some suggestions, answering this might be a good way to earn some reputation. ;-) – Cragsman

hc.apache.org/httpcomponents-client-ga/tutorial/html/… – Unread

Answer 1

Much of the performance hit of http connections is establishing the socket connection. You can avoid this by using 'keep-alive' http connections. To do this, it's best to use HTTP 1.1 and make sure that "Content-Length: xx" is always set in requests and responses, "Connecction: close" is correctly set when appropriate and is properly acted upon when received.

Unidirectional