Use java http connection instance after exception

Question

Is the following code safe:

try {
    URL url = new URL(urlRequest);
    conn = (HttpURLConnection)url.openConnection();
    conn.setConnectTimeout(30000);
    conn.setReadTimeout(30000);
    conn.setRequestProperty("Accept-Encoding", "gzip, deflate");
    String encoding = conn.getContentEncoding();
    return Utils.wrapCompressedStream(conn.getInputStream(), encoding);
} catch (IOException e) {
    if(conn != null) {
        conn.getContentEncoding();
        conn.getErrorStream();
        conn.whateverOtherMethodThere();
        ...
    }
}

Particularly, is it safe in case of InterruptedIOException (say, read timeout) to call methods like getContentEncoding()? As far as I understand this method requires live connection to read HTTP(S) headers.

Update (additional info):

This question originates from a real-system experience. I believe, the system was run on Oracle/Sun JVM 1.6 then. The code is almost the same:

...    
} catch (IOException e) {
    if(conn != null) {
         try {
             String response = tryGetResponse(conn);
...

The problem happened in the tryGetResponse on HTTPS requests:

 private static String tryGetResponse(HttpURLConnection conn) {
    if(conn == null) return "(failed to get)";
    InputStream in = null;
    try {
        InputStream err = conn.getErrorStream();
        if (err != null) {
            in = Utils.wrapCompressedStream(err, conn.getContentEncoding());
        }
        return Utils.inputStreamToString(in);
    } catch (IOException e) {
        return "(failed to get)";
    } finally {
        Utils.closeQuitely(in);
    }
}

Spontaneously system hanged on the socket connect (or read) in the getContentEncoding() call:

in = Utils.wrapCompressedStream(err, conn.getContentEncoding());

exactly after SocketTimeoutException is thrown in the initial code.

So, it seems that getContentEncoding() tries (or tried in Java 6) to establish a new connection without timeouts being set.

Elenore

No, it is not, if you try to get a header field like content encoding from an incomplete inputstream you will get another IOException, check the HttpURLConnection source code. – Digitalis

Answer 1

No. It is not safe in general. The behavior might differ between JVM implementations (think IBM J9 vs. Oracle VM vs. Open JDK) and change between versions of the same VM without notice.

The reason for this is that the API specification makes no guarantees.

If you tell me which specific implementation in which particular version you are using, I could look in the sources and try to make some conclusions. But I would strongly advise against relying about the behavior you find there.

With regards to HTTPS: There seem still to be some bugs with SSL. For example OpenSSL has made a public announcement, that they will reveal a security bug end of this week. The bug fixes for that might change the behavior of a HTTPS connection in some error cases. It's possible that whatever we find in the sources will be moot at the end of this week. If not, it might change with the next security fix.

Update:

I've tried to find the sources that correspond to the version of Java you have referenced in your updated question. Finding the Java-Sources is not a big problem, but the code goes very quickly into native parts. This alone is a good hint, that the answer would not only be version specific, but also platform specfic (Linux, Windows, Mac, etc). You can check yourself on the OpenJDK sources for Java 1.6, e.g. for the network stack for Windows.

Beware: You have very likely used the Sun JDK 1.6. OpenJDK 1.6 is besed on the Sun/Oracle JDK, but on JDK 1.7. The Open JDK 1.6 is code form Sun/Oracle JDK 1.7 backportet to Java 1.6. So there still might be some small differences, that nevertheless can be significant regarding the usage of the connection after an error occured.

Mcmath

+1 Your answer is very valuable as sometimes it may be so that developers work on a JVM implementation and the program runs on a different one. On the other side, EJP's answer is the actual solution: you MUST deal with any JVM implementation in the same code, by simply trying and catching. – Ruminant

@Mcmath I've updated the question to include more details. – Elenore

Answer 2

For getErrorStream the spec says:

Returns the error stream if the connection failed but the server sent useful data nonetheless. The typical example is when an HTTP server responds with a 404, which will cause a FileNotFoundException to be thrown in connect, but the server sent an HTML help page with suggestions as to what to do. This method will not cause a connection to be initiated. If the connection was not connected, or if the server did not have an error while connecting or if the server had an error but no error data was sent, this method will return null. This is the default.

Returns: an error stream if any, null if there have been no errors, the connection is not connected or the server sent no useful data.

You can also inspect the source to clear up any doubts, so lets look at some of the methods:

public InputStream getErrorStream() {
    if (connected && responseCode >= 400) {
        // Client Error 4xx and Server Error 5xx
        if (errorStream != null) {
            return errorStream;
        } else if (inputStream != null) {
            return inputStream;
        }
    }
    return null;
}

It is safe to call this (no exception will be thrown) and the errorStream can be set in some IOException cases. The comment in the source is buffer the error stream if bytes < 4k and it can be buffered within 1 second.

getContentEncodings behaviour from the spec is:

Returns: the content encoding of the resource that the URL references, or null if not known.

But what about after an error? Lets look at code:

public String getContentEncoding() { //from the base class java.net.URLConnection
    return getHeaderField("content-encoding");
}

public String getHeaderField(String name) {
    try {
        getInputStream();
    } catch (IOException e) {} //ah exception is eaten

    if (cachedHeaders != null) {
        return cachedHeaders.findValue(name);
    }

    return responses.findValue(name);
}

So it caches headers and returns them if known, even potentially after an IOException, it doesn't propagate an exception but might return null if it has not previously got the headers successfully.

Holding

The specification is a greater authority than the source. It says what shouldl happen. The source may contain bugs, or implementation details you shouldn't rely on. – Metzgar

There is no guessing in my answer. As a matter of fact it agrees with what you've stated in your answer above. – Metzgar

@EJP Sorry, you're correct the errorStream can be set in some cases. – Holding

@EJP You're totally right, my two conclusions from a quick skim of the code were both wrong and contradictory to the spec! – Holding

@EJP For the completeness of your comment: I definitely agree that the specification is the greatest authority in Java and everything that does not implement the specification may and MUST be considered a bug. But as no document is perfect, there are parts that are not covered by the spec, in which case we rely on the source code. – Ruminant

@AndreiI That's just self-contradictory. You can't rely on it if it may be a bug. It might get fixed, or it might not even be present in another implementation. – Metzgar

@EJP I would agree with you, if there would be a better solution. Usually we have to deliver, deliver and again to deliver, so I would write a Unit Test for exactly that part (the bug), so that when the bug is repaired, my unit test forces me to correct my code. But waiting for a bug to be corrected does not seem to me like a solution. – Ruminant

Answer 3

Particularly, is it safe in case of InterruptedIOException (say, read timeout) to call methods like getContentEncoding()?

You can try. The worst that will happen is anotherIOException. In several cases of IOException, for example FileNotFoundException, it is perfectly safe to call getErrorStream() and read the content of an error page.

Metzgar

I tried, actually. System hanged a lot on https requests. I've updated my question to include more details. Still, don't sure if that behaviour a Java bug or it's generally not safe to call getContentEncoding (after timeouts only, when the connection is definitely dead?). – Elenore

If you get a read timeout, (a) the content is short, (b) there is possibly no content at all, and (c) there are possibly no headers either. I don't understand why you think there's a bug here. – Metzgar

What about connect timeout? It hangs on getContentEncoding when you try to read error stream. – Elenore

I'd have to see the stack trace of that. – Metzgar

Answer 4

catch (IOException e) {
    if(conn != null) {
        conn.getContentEncoding();
        conn.getErrorStream();
        conn.whateverOtherMethodThere();
        ...
    }

public class InterruptedIOException extends IOException

Signals that an I/O operation has been interrupted. An InterruptedIOException is thrown to indicate that an input or output transfer has been terminated because the thread performing it was interrupted. The field bytesTransferred indicates how many bytes were successfully transferred before the interruption occurred.

That means if an interrupted exception occurs you can not do further processing on the HttpUrlConnection.

Again Through IOException you can not catch other type of exception like IllegalArgumentException, illegalthreadstate exception and many more.

For more details http://download.java.net/jdk7/archive/b123/docs/api/java/net/HttpURLConnection.html#getRequestMethod%28%29

Racial

I don't do any processing on the socket itself, I work with the HttpUrlConnection instance. And the question is - is it safe to work with HttpUrlConnection, not with socket. – Elenore

sorry, i have update by answer. i have missed the keyword. it will be HttpUrlConnection. – Racial

Is it specified somewhere? If I can't use socket it doesn't mean I can't use UrlConnection either. For example, UrlConnection may throw exception or return null after InterruptedException was thrown. E.g. obviously it's safe to use getErrorStream. – Elenore

It does not mean that you can't keep using the connection. It means that whatever you were doing was interrupted. At a minmum you must certainly close whatever streams you have acquired from the connection, which constitutes using it, or disconnect it, ditto. What the Javadoc of getRequestMethod() has to do with it is a total mystery. It's just a getter for a local variable. No network operations expressed or implied, and no statement anywhere that agrees with what you've claimed here. The part about IllegalArgumentException etc. is totally irrelevant. -1 – Metzgar