How to get error information when HttpWebRequest.GetResponse() fails

Question

I am initiating an HttpWebRequest and then retrieving it's response. Occasionally, I get a 500 (or at least 5##) error, but no description. I have control over both endpoints and would like the receiving end to get a little bit more information. For example, I would like to pass the exception message from server to client. Is this possible using HttpWebRequest and HttpWebResponse?

Code:

try
{
    HttpWebRequest webRequest = HttpWebRequest.Create(URL) as HttpWebRequest;
    webRequest.Method = WebRequestMethods.Http.Get;
    webRequest.Credentials = new NetworkCredential(Username, Password);
    webRequest.ContentType = "application/x-www-form-urlencoded";
    using(HttpWebResponse response = webRequest.GetResponse() as HttpWebResponse)
    {
        if(response.StatusCode == HttpStatusCode.OK)
        {
            // Do stuff with response.GetResponseStream();
        }
    }
}
catch(Exception ex)
{
    ShowError(ex);
    // if the server returns a 500 error than the webRequest.GetResponse() method
    // throws an exception and all I get is "The remote server returned an error: (500)."
}

Any help with this would be much appreciated.

Codling

I'll just add to this, it's always recommended to minimize the content wrapped by a try statement. In your case, everything up to the using line could probably be written outside. – Humanism

Answer 1

Is this possible using HttpWebRequest and HttpWebResponse?

You could have your web server simply catch and write the exception text into the body of the response, then set status code to 500. Now the client would throw an exception when it encounters a 500 error but you could read the response stream and fetch the message of the exception.

So you could catch a WebException which is what will be thrown if a non 200 status code is returned from the server and read its body:

catch (WebException ex)
{
    using (var stream = ex.Response.GetResponseStream())
    using (var reader = new StreamReader(stream))
    {
        Console.WriteLine(reader.ReadToEnd());
    }
}
catch (Exception ex)
{
    // Something more serious happened
    // like for example you don't have network access
    // we cannot talk about a server exception here as
    // the server probably was never reached
}

Barajas

Thank you! Important to note that the stream from inside the using statement will not be available outside the using statement, as WebResponse's disposer will clear it. This tripped me up for a few minutes. – Adorn

@Thorin. The "stream" from the first statement is carried on to the next statement. Just like in a single line IF statement, for example if(something)do-stuff-here; – Goldenrod

GetRequestStream and GetResponse can throw exceptions ? – Silent

@PreguntonCojoneroCabrón Yea, it doesn't seem quite right, does it. Thankfully Microsoft introduced the HttpClient-class, that I suspect most are using these days. msdn.microsoft.com/en-us/library/… – Tempura

WebException populates Response only when Status==WebExceptionStatus.ProtocolError. Most errors will not let you access the response body from the server and Response will be null. – Hawsepiece

Answer 2

I came across this question when trying to check if a file existed on an FTP site or not. If the file doesn't exist there will be an error when trying to check its timestamp. But I want to make sure the error is not something else, by checking its type.

The Response property on WebException will be of type FtpWebResponse on which you can check its StatusCode property to see which FTP error you have.

Here's the code I ended up with:

    public static bool FileExists(string host, string username, string password, string filename)
    {
        // create FTP request
        FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://" + host + "/" + filename);
        request.Credentials = new NetworkCredential(username, password);

        // we want to get date stamp - to see if the file exists
        request.Method = WebRequestMethods.Ftp.GetDateTimestamp;

        try
        {
            FtpWebResponse response = (FtpWebResponse)request.GetResponse();
            var lastModified = response.LastModified;

            // if we get the last modified date then the file exists
            return true;
        }
        catch (WebException ex)
        {
            var ftpResponse = (FtpWebResponse)ex.Response;

            // if the status code is 'file unavailable' then the file doesn't exist
            // may be different depending upon FTP server software
            if (ftpResponse.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
            {
                return false;
            }

            // some other error - like maybe internet is down
            throw;
        }
    }

Ana

Answer 3

I faced a similar situation:

I was trying to read raw response in case of an HTTP error consuming a SOAP service, using BasicHTTPBinding.

However, when reading the response using GetResponseStream(), got the error:

Stream not readable

So, this code worked for me:

try
{
    response = basicHTTPBindingClient.CallOperation(request);
}
catch (ProtocolException exception)
{
    var webException = exception.InnerException as WebException;
    var rawResponse = string.Empty;

    var alreadyClosedStream = webException.Response.GetResponseStream() as MemoryStream;
    using (var brandNewStream = new MemoryStream(alreadyClosedStream.ToArray()))
    using (var reader = new StreamReader(brandNewStream))
        rawResponse = reader.ReadToEnd();
}

Phrase

Answer 4

You can also use this library which wraps HttpWebRequest and Response into simple methods that return objects based on the results. It uses some of the techniques described in these answers and has plenty of code inspired by answers from this and similar threads. It automatically catches any exceptions, seeks to abstract as much boiler plate code needed to make these web requests as possible, and automatically deserializes the response object.

An example of what your code would look like using this wrapper is as simple as

    var response = httpClient.Get<SomeResponseObject>(request);
    
    if(response.StatusCode == HttpStatusCode.OK)
    {
        //do something with the response
        console.Writeline(response.Body.Id); //where the body param matches the object you pass in as an anonymous type.  
    }else {
         //do something with the error
         console.Writelint(string.Format("{0}: {1}", response.StatusCode.ToString(), response.ErrorMessage);

    }

Full disclosure This library is a free open source wrapper library, and I am the author of said library. I make no money off of this but have found it immensely useful over the years and am sure anyone who is still using the HttpWebRequest / HttpWebResponse classes will too.

It is not a silver bullet but supports get, post, delete with both async and non-async for get and post as well as JSON or XML requests and responses. It is being actively maintained as of 6/21/2020

Whitehorse

Answer 5

Sometimes ex.Response also throws NullReferenceException so below is the best way to handle

catch (WebException ex)
{
    using (var stream = ex?.Response?.GetResponseStream())
    if(stream != null)
    using (var reader = new StreamReader(stream))
    {
        Console.WriteLine(reader.ReadToEnd());
    }
    // todo...
}
catch (Exception ex)
{
    // todo...
}

Sabulous

Answer 6

**Answer Updated on 14-03-2022**
HttpWebRequest myHttprequest = null;
HttpWebResponse myHttpresponse = null;
try
{
myHttpRequest = (HttpWebRequest)WebRequest.Create(URL);
myHttpRequest.Method = "POST";
myHttpRequest.ContentType = "application/x-www-form-urlencoded";
myHttpRequest.ContentLength = urinfo.Length;
StreamWriter writer = new StreamWriter(myHttprequest.GetRequestStream());
writer.Write(urinfo);
writer.Close();
myHttpresponse = (HttpWebResponse)myHttpRequest.GetResponse();
if (myHttpresponse.StatusCode == HttpStatusCode.OK)
 {
   //Success code flow
 }
myHttpresponse.Close(); 
}
catch(WebException e) {
    Console.WriteLine("This program is expected to throw WebException on successful run."+
                    "\n\nException Message :" + e.Message);
    if(e.Status == WebExceptionStatus.ProtocolError) {
        Console.WriteLine("Status Code : {0}", 
    ((HttpWebResponse)e.Response).StatusCode);
    Console.WriteLine("Status Description : {0}", 
    ((HttpWebResponse)e.Response).StatusDescription);
}
**Updated Answer with try catch block**
[docs.microsoft][1]

Marguritemargy