HttpWebRequest.GetResponse throws WebException on HTTP 304

Question

When a web server responds to HttpWebRequest.GetResponse() with HTTP 304 (Not Modified), GetResponse() thows a WebException, which is so very weird to me. Is this by design or am I missing something obvious here?

Crista

Answer 1

Ok, this seems to be a by-design behavior and a perfect example of a vexing exception. This can be solved with this:

public static HttpWebResponse GetHttpResponse(this HttpWebRequest request)
{
    try
    {
        return (HttpWebResponse) request.GetResponse();
    }
    catch (WebException ex)
    {
        if(ex.Response == null || ex.Status != WebExceptionStatus.ProtocolError)
            throw; 

        return (HttpWebResponse)ex.Response;
    }
}

Crista

This works most of the cases, but some web servers could return a response body when returning a 404 error. In that case, the code above would treat a 404 as it treats a 304! – Cleanthes

@Cleanthes that's a "good to know." The calling code will have to be aware of what are acceptable response codes. – Gisser

I also added || ((HttpWebResponse) ex.Response).StatusCode != HttpStatusCode.NotModified – Brindabrindell

You should update this to use C#-6.0’s when clause! – Coniferous

As a bonus, one used to be able to decorate a method with [DebuggerNonUserCode] and the Debugger would not stop in that method when an exception is thrown. In this manner poorly designed exceptions could be wrapped and ignored. But now a registry setting is required – Kif

Answer 2

This is really a frustrating problem, and can be alternatively worked around by using the following extension method class and calling request.BetterGetResponse()

//-----------------------------------------------------------------------
//
//     Copyright (c) 2011 Garrett Serack. All rights reserved.
//
//
//     The software is licensed under the Apache 2.0 License (the "License")
//     You may not use the software except in compliance with the License.
//
//-----------------------------------------------------------------------

namespace CoApp.Toolkit.Extensions {
    using System;
    using System.Net;

    public static class WebRequestExtensions {
        public static WebResponse BetterEndGetResponse(this WebRequest request, IAsyncResult asyncResult) {
            try {
                return request.EndGetResponse(asyncResult);
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }

        public static WebResponse BetterGetResponse(this WebRequest request) {
            try {
                return request.GetResponse();
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }
    }
}

You read more about it in my blog post on this subject at http://fearthecowboy.com/2011/09/02/fixing-webrequests-desire-to-throw-exceptions-instead-of-returning-status/

Pyroxenite

Answer 3

The way to avoid this System.WebException is to set AllowAutoRedirect property to false. This disables the automatic redirection logic of the WebRequest. It seems to be broken for 304 redirection requests, as it is not a real redirection in the strictest sense. Of course that means that the other redirection requests 3xx have to be handled manually.

Widower

Absolutely brilliant. Why should I pay for the heavy-handed exception machinery if I don't need it? – Condescending

Answer 4

Just as an FYI, this is an update to Anton Gogolev's answer that uses the C#6 (VS2015) when clause. It's a little less annoying when using a debugger as it removes one catchpoint:

public static HttpWebResponse GetHttpResponse(this HttpWebRequest request)
{
    try
    {
        return (HttpWebResponse) request.GetResponse();
    }
    catch (WebException ex)
        when (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
    {
        return (HttpWebResponse) ex.Response;
    }
}

Andyane

Answer 5

I also came across to this issue with code:

try
{
    ...
    var webResponse = req.GetResponse();
    ...
}
catch (WebException ex)
{
    Log.Error("Unknown error occured", ex);
    //throw; 
}

And it appears that if Remote Server returns 304 status it must be passed to Browser by throwing this error or returning custom 304 so the Browser could return cached response. Otherwise you will probably get empty Response from the Remote Server.

So in my case for normal behaviour with correct Cache handling it should be like:

try
{
    ...
    var webResponse = req.GetResponse();
    ...
}
catch (WebException ex)
{
    if (((HttpWebResponse)ex.Response).StatusCode == HttpStatusCode.NotModified)
        throw;
    Log.Error("Unknown error occured", ex);
}

Discontent