Android (Java) HttpURLConnection silent retry on 'read' timeout

Question

So I'm using Google Volley for HTTP request, which basically uses Java's HttpURLConnection.

According to my tests, the problem is this:
When the 'read' timeout on the HttpURLConnection reaches, a silent retry is executed before the connection is closed and the relevant exception thrown (SocketTimeoutException).

Note that:
- I noticed this error when using HTTP POST request.
- 'read' timeout is different than 'connect' timeout.
- If the 'read' timeout (set by calling connection.setReadTimeout(int)) is NOT set (0), or set to a greater value than connection.setConnectTimeout(int), this error does not occur.
- This issue has been discussed, here for example, but I haven't found any satisfying solution.
- A somewhat related issue can be found here, but I'm not sure it is relevant (is it?)

More Background
My app is used for paying money, so not retrying a request is crucial (yes, I know it can be handled by the server, I want my client to be "correct" anyway).

When the 'read' timeout is set, in case the server connection is established, but the server waits/sleeps/delays-response that 'timeout' time before answering (thus raising the 'read' exception and not the 'connect' exception), another (silent) request is sent just before that exception is raised, resulting in 2 similar requests, which is not acceptable.

What kind of solution am I looking for?
Well, a one that will nicely solve this problem/bug, just like the fix explained here (but I again, I think it's irrelevant in this case).
Also, I would want to keep the original flow as is, meaning not forcing the connection to close or anything like that.

What I'm gonna do for now, is set the 'read' timeout to twice the 'connection' timeout (they start counting at the same time), to make sure the 'connection' exception is raised first. I will also try to overcome this issue on the server side. The problem is, this 'read' timeout is there for a reason, and my current implementation practically just ignores it, and handles only 'connection' timeouts.

EDIT The Volley library's RetryPolicy has not affect on this issue, as this is a silent retry. I looked as deep as possible inside the library. Logs/breakpoints everywhere, cancelled the calls to retry. That how I know it is 99.99% a HttpURLConnection issue.

Amphiaster

were you able to solve it? – Menstruation

Not a perfect solution, but as I said, when I'm doing now, is set the 'read' timeout to equal or twice the 'connection' timeout. I'll answer myself for other people to easily see this solution. – Amphiaster

Answer 1

This bad decision was made by a developer in the year 2006. Here is a nice citation from someone that explains the whole situation from java perspective:

"As you probably guessed by now it's a bug (http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6382788). Not the retry mechanism, of course, that's just crap. The bug is that it also happens for a POST (which is by default not idempotent per HTTP RFC). But don't worry, Bill fixed that bug a long time ago. Bill fixed it by introducing a toggle. Bill learned about backward compatibility. Bill decided it's better to leave the toggle 'on' by default because that would make it bug-backward-compatible. Bill smiles. He can already see the faces of surprised developers around the globe running into this. Please don't be like Bill?"
Source

Well the proposed solution is:

System.setProperty("sun.net.http.retryPost", "false")

But we cannot do this on android! The only remaining solution then is:

httpURLConnection.setChunkedStreamingMode(0);

Which seems to be working, but isn't very effective from a requesting-time perspektive.

EDIT: I could not use this implementation so i looked for an alternative library. I found out that the implementation of HttpUrlConnection uses OkHttp since Android 4.4. As OkHttp is opensource i could search if they have also problems with silent retries. And yep they had problems with it and fixed it in April, 2016. CommonsWare (a real android expert) explains that every manufacturer can decide which implementation they can use. As a consequence of this there must be a lot of devices out there that do silent retries on POST requests and as a developer we only can try some workarrounds.

My solution will be to change the library

EDIT 2: To give you a final answer: You can now use OkHttp as the transport layer for Volley with minimal code.

Another helpful solution

Normi

So... httpURLConnection.setChunkedStreamingMode(0) will result in no retries? Please explain the effect in the answer:) And yeah, totally the kind of answer I would've liked to hear... – Amphiaster

I had problems with the proposed solution, so i edited the answer. – Normi

Please add a link to my answer below. I suppose it can still help others... thx. – Amphiaster

another solution is to use setFixedLengthStreamingMode(contentLength) which solves your requesting time perspective. read this issuetracker.google.com/issues/37036206 – Bargainbasement

Answer 2

Well, lot of time passed so I figured I will answer with my current (not perfect) solution for other people to easily see it (also written inside the question).

Just set the readTimeout to 0 (no timeout) or at least the connection timeout value. This will make sure connection timeout will be raised before the read timeout.

int timeoutMs = request.getTimeoutMs();    //  or whatever
connection.setConnectTimeout(timeoutMs);   //  actual timeout desired
connection.setReadTimeout(timeoutMs);      //  >= timeoutMs or 0

Of course, it depends on the use of the request. Most of the times, the app is part of a project containing the related server, and you know read timeouts shouldn't happen anyway..

As said, not really a "solution", but a very nice and 90% useful fix.

Amphiaster

Answer 3

You can check out DefaultRetryPolicy class of Volley.

Wart

Thx, but this issue is deeper. The first thing I checked was the retry policy, and I did cancel it. This retry is silent so Volley doesn't event know about it. – Amphiaster

Answer 4

As it says in one of the links you quoted, you need to set sun.net.http.retryPost to false.

Or, don't use fixed-length or chunked transfer mode: see this bug report.

Prochronism

I can't find sun as a dependency though, any ideas? – Amphiaster