My application will send approx 1000 POST requests/minute to my tomcat server(which is http/2 enabled) which will poll the given url and return back the html and response time, I want to achieve true http/2 multiplexing to reuse tcp connection between my application and tomcat server. my client uses okhttp and I can successfully establish connection and reuse it for a long time, but when the number of request count increases(something like, queued:50, running:50), the response time also increases(it spikes to 2000ms-15000ms or even worse, which would normally take 300ms-500ms). I can understand this is happening because of overloading of tcp connection with too many requests, So I have decided to open multiple tcp connections and allow it to distribute the request load across tcp connections. I force the client to open multiple connections using
Dispatcher dispatcher = new Dispatcher();
dispatcher.setMaxRequests(100);
dispatcher.setMaxRequestsPerHost(5);
ConnectionPool cp = new ConnectionPool(5, 5, TimeUnit.MINUTES);
With the help of wireshark, I can see 5 connections opened and also see 4 of the requests are closed immediately after the first handshake is succeeded, I don't know whether that's the expected behaviour of http/2 multiplexing.
If that's expected behaviour, how can I optimize it to reduce the response time of each request at peak load time?
Or is that possible to use multiple connections to distribute the load at peak load time?
My sample program as follows,
Security.insertProviderAt(Conscrypt.newProvider(), 1);
sslContext = SSLContext.getInstance("TLSv1.3");
sslContext.init(null, new TrustManager[] {
new X509TrustManager() {
@Override
public void checkClientTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
@Override
public void checkServerTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
@Override
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new java.security.cert.X509Certificate[]{};
}
}
}, new java.security.SecureRandom());
sslSocketFactory = sslContext.getSocketFactory();
Dispatcher dispatcher = new Dispatcher();
dispatcher.setMaxRequests(100);
dispatcher.setMaxRequestsPerHost(5);
ConnectionPool cp = new ConnectionPool(5, 1, TimeUnit.DAYS);
okHttpClient = new OkHttpClient().newBuilder()
.sslSocketFactory(sslSocketFactory, (X509TrustManager)trustManager[0])
.dispatcher(dispatcher)
.connectionPool(cp)
.hostnameVerifier(new HostnameVerifier() {
@Override
public boolean verify(String hostname, SSLSession session) {
return true;
}
})
.build();
We use conscryt to enable ALPN in jdk8 itself,
try {
String url = "https://localhost:8081/myAent/getOutput?1000000004286_1520339351002_"+System.currentTimeMillis();
String json = "<?xml version=\"1.0\" standalone=\"no\"?><UC mid=\"1000000005011\" pollNow=\"true\" use_ipv6=\"false\" rca=\"0\" rcaFN=\"P|TA|D|J|M\" pFtct=\"\" sFtct=\"\" issecondarydc=\"false\" ssDc=\"false\" nocache=\"1\" storeHtmlResp=\"true\" storeTroubleScrnSht=\"false\" moConfig=\"false\" xconlen=\"8265\" noScreenshotRecheckForSSLErrors=\"true\" avgDnsTime=\"null\" isProxyRequired=\"false\" userroles=\"EVAL_USER\" uid=\"102030230293029021\" retryDelay=\"2\" retry=\"true\" idcLocUrl=\"http://localhost:8080/app/receivemultipartdata\" isRemoteAgent=\"true\" sendHeaders=\"false\" api=\"ab_345nnn4l4lj4lk23nl4k23342lk4l23j4\" ut=\""+(System.currentTimeMillis()+"")+"\" mt=\"URL\" dctimeout=\"70\" pollinterval=\"1440\" locid=\"48\" log=\"1\" currentstatus=\"1\" postUrl=\"https://example.com\"><Url acc=\"\" forced_ips=\"\" use_ipv6=\"false\" client_cert=\"\" mid=\"1000000005011\" sotimeout=\"60\" ua=\"\" ds=\"117.20.43.94\" ucc=\"\" md=\"false\" client_cert_pass=\"\" context=\"default\" unavail_alert=\"\" ssl_protocol=\"\" avail_alert=\"\" enabledns=\"false\" enableBouncyCastle=\"false\" cc=\"false\" a=\"https://www.example.com/tools.html\" upStatusCodes=\"\" regex_alert=\"\" probeproxy=\"false\" m=\"G\" keyword_case=\"0\" regex=\"\" rbc=\"\" t=\"30\" lc=\"English\"><PD></PD><CH hn=\"\" hd=\"_sep_\" hv=\"\"/><AI ps=\"\" un=\"\"/></Url></UC>";
RequestBody body = RequestBody.create(MediaType.get("application/json; charset=utf-8"), json);
Request request = new Request.Builder()
.url(url)
.post(body)
.build();
long nanoStartTime = System.nanoTime();
okHttpClient.newCall(request).enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
System.out.println("okhttp3:: Request failed"+ e);
}
@Override
public void onResponse(Call call, okhttp3.Response responseObj) throws IOException {
try (ResponseBody body = responseObj.body()) {
long nanoEndTime = System.nanoTime();
long nanoDiffTime = TimeUnit.NANOSECONDS.toMillis(nanoEndTime - nanoStartTime);
System.out.println("okhttp3:: Succeded response ***"+body+"$$$");
System.out.println("okhttp3:: Request Succeded protocol ***"+responseObj.protocol()+"$$$, time is "+nanoDiffTime);
}
}
});
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
How can I optimize okhttpclient to use multiple tcp sockets/connection for achieveing http/2 multiplexing to distribute the request load.
Client : Tomcat apache - 9.0.x, Jdk - 8, Http library - Okhttp3, Os - Ubuntu/Centos, Security Provider - Conscrypt(For supporting ALPN in jdk 8).
Server : Tomcat apache - 9.0.16, Jdk -10.0.1, Os - Ubuntu/Centos, OpenSSL - 1.1.1a for TLSv1.3 support