.Net HttpWebRequest.GetResponse() raises exception when http status code 400 (bad request) is returned
Asked Answered
P

7

230

I am in a situation where when I get an HTTP 400 code from the server, it is a completely legal way of the server telling me what was wrong with my request (using a message in the HTTP response content)

However, the .NET HttpWebRequest raises an exception when the status code is 400.

How do I handle this? For me a 400 is completely legal, and rather helpful. The HTTP content has some important information but the exception throws me off my path.

Proboscidean answered 28/3, 2009 at 6:40 Comment(1)
I was experiencing the same thing. I submitted a suggestion to the .NET Framework team. Feel free to vote for it: connect.microsoft.com/VisualStudio/feedback/details/575075/…Quiff
F
404

It would be nice if there were some way of turning off "throw on non-success code" but if you catch WebException you can at least use the response (if there is one):

using System;
using System.IO;
using System.Web;
using System.Net;

public class Test
{
    static void Main()
    {
        WebRequest request = WebRequest.Create("http://csharpindepth.com/asd");
        try
        {
            using (WebResponse response = request.GetResponse())
            {
                Console.WriteLine("Won't get here");
            }
        }
        catch (WebException e)
        {
            using (WebResponse response = e.Response)
            {
                // TODO: Handle response being null
                HttpWebResponse httpResponse = (HttpWebResponse) response;
                Console.WriteLine("Error code: {0}", httpResponse.StatusCode);
                using (Stream data = response.GetResponseStream())
                using (var reader = new StreamReader(data))
                {
                    string text = reader.ReadToEnd();
                    Console.WriteLine(text);
                }
            }
        }
    }
}

You might like to encapsulate the "get me a response even if it's not a success code" bit in a separate method. (I'd suggest you still throw if there isn't a response, e.g. if you couldn't connect.)

If the error response may be large (which is unusual) you may want to tweak HttpWebRequest.DefaultMaximumErrorResponseLength to make sure you get the whole error.

Fushih answered 28/3, 2009 at 7:7 Comment(17)
The contents of the stream returned by GetResponseStream() on the response attached to the WebException is just the name of the status code (e.g. "Bad Request") rather than the response actually returned by the server. Is there any way to get this information?Brach
@MarkWatts: It should be whatever's been returned by the server, and has been in every situation I've seen. Can you reproduce this with a particular external URL? I suggest you ask a new question (referring to this one) and showing what's going on.Fushih
Turns out that it only does this when the content length of the response is zero; it adds a textual description of the HTTP status code - 400 is just "Bad Request" but some of the others are more descriptive.Brach
If anybody knows of a nice wrapper for this class drop a link to it here. System.Net.WebClient behaves the same way by invoking the exception handling system.Adaptable
If you have a large response it seems not to read all response contentUnchaste
@Ghita: If all the content is in the stream, it will read it all. If you're seeing something else, you should create a new question with more details.Fushih
@JonSkeet you must set HttpWebRequest.DefaultMaximumErrorResponseLength in order to control the response stream length that is available. By default the length read for error case could be too short.Unchaste
@JonSkeet Does a web request automatically throw WebException if response code is not 200 or 201? or is it possible that we may not get exception even when status code is 400?Voltage
@AnkushJain: I believe it will return normally for 2XX; redirection may happen for 3XX depending on configuration - I'd expect anything else to cause an exception, though I could be wrong. (For 4XX, it's just possible that it will then apply auth information if it has it but hadn't already used it.)Fushih
As a bonus, one used to be able to decorate a method with [DebuggerNonUserCode] and the Debugger would not stop in that method when an exception is thrown. In this manner poorly designed exceptions could be wrapped and ignored. But now a registry setting is requiredCysticercus
Not try-catch for Stream data = response.GetResponseStream() ?Rosenberry
@PreguntonCojoneroCabrón So the exception wouldn't be caught... I'm not sure what point you're trying to make.Fushih
In your code sample not try-catch for using (Stream data = response.GetResponseStream()) code fragmentRosenberry
@PreguntonCojoneroCabrón Indeed - if an exception is thrown, what could I do about it? Just print the message, which is what will happen anyway. If you find you've got catch blocks all over your code, you may want to rethink your error handling strategy.Fushih
You saved me. Details here #65543063 Thank you so muchSloatman
A WebException does not always contain a response. For a WebException Status other than WebExceptionStatus.ProtocolError, Response will be null. That means there is no way to capture a server response that isn't a "protocol error", like a 504 Gateway Timeout that has a response body.Cyrenaica
@Suncat2000: Edited (briefly) to indicate that.Fushih
P
53

I know this has already been answered a long time ago, but I made an extension method to hopefully help other people that come to this question.

Code:

public static class WebRequestExtensions
{
    public static WebResponse GetResponseWithoutException(this WebRequest request)
    {
        if (request == null)
        {
            throw new ArgumentNullException("request");
        }

        try
        {
            return request.GetResponse();
        }
        catch (WebException e)
        {
            if (e.Response == null)
            {
                throw;
            }

            return e.Response;
        }
    }
}

Usage:

var request = (HttpWebRequest)WebRequest.CreateHttp("http://invalidurl.com");

//... (initialize more fields)

using (var response = (HttpWebResponse)request.GetResponseWithoutException())
{
    Console.WriteLine("I got Http Status Code: {0}", response.StatusCode);
}
Physoclistous answered 17/1, 2013 at 17:56 Comment(5)
WebException.Response can and may be null. You should rethrow if this is the case.Tratner
@DavidP I agree, the usage is a little janky, though for most things, you should probably be using HttpClient instead, it's much more configurable, and I believe it's the way of the future.Physoclistous
Is checking for request == null really necessary? Since this is an extension method, trying to use it on a null object should throw a null reference exception before it hits the extension method code......right?Christchurch
@Christchurch Extension methods are just static methods, doing proper input validation should be done to avoid confusion, especially when they're not invoked with the extension method syntax. Also, this is the consistent approach done by the the .NET teams: github.com/dotnet/corefx/blob/…Physoclistous
Sorry I misunderstood your second question. ((WebRequest) null).GetResponseWithoutException() will in fact not cause a NullReferenceException, since it is compiled to the equivalent of WebRequestExtensions.GetResponseWithoutException(null), which would not result in a NullReferenceException, hence the need for input validation.Physoclistous
S
15

Interestingly, the HttpWebResponse.GetResponseStream() that you get from the WebException.Response is not the same as the response stream that you would have received from server. In our environment, we're losing actual server responses when a 400 HTTP status code is returned back to the client using the HttpWebRequest/HttpWebResponse objects. From what we've seen, the response stream associated with the WebException's HttpWebResponse is generated at the client and does not include any of the response body from the server. Very frustrating, as we want to message back to the client the reason for the bad request.

Speciality answered 20/4, 2010 at 15:29 Comment(2)
I use method HEAD and cause this exception but when use GET there is no any problem. What is the issue with HEAD method exactly?Didymium
@MohammadAfrashteh HEAD doesn't include a message body in the response; therefore, no response stream is available.Cyrenaica
I
13

I had similar issues when trying to connect to Google's OAuth2 service.

I ended up writing the POST manually, not using WebRequest, like this:

TcpClient client = new TcpClient("accounts.google.com", 443);
Stream netStream = client.GetStream();
SslStream sslStream = new SslStream(netStream);
sslStream.AuthenticateAsClient("accounts.google.com");

{
    byte[] contentAsBytes = Encoding.ASCII.GetBytes(content.ToString());

    StringBuilder msg = new StringBuilder();
    msg.AppendLine("POST /o/oauth2/token HTTP/1.1");
    msg.AppendLine("Host: accounts.google.com");
    msg.AppendLine("Content-Type: application/x-www-form-urlencoded");
    msg.AppendLine("Content-Length: " + contentAsBytes.Length.ToString());
    msg.AppendLine("");
    Debug.WriteLine("Request");
    Debug.WriteLine(msg.ToString());
    Debug.WriteLine(content.ToString());

    byte[] headerAsBytes = Encoding.ASCII.GetBytes(msg.ToString());
    sslStream.Write(headerAsBytes);
    sslStream.Write(contentAsBytes);
}

Debug.WriteLine("Response");

StreamReader reader = new StreamReader(sslStream);
while (true)
{  // Print the response line by line to the debug stream for inspection.
    string line = reader.ReadLine();
    if (line == null) break;
    Debug.WriteLine(line);
}

The response that gets written to the response stream contains the specific error text that you're after.

In particular, my problem was that I was putting endlines between url-encoded data pieces. When I took them out, everything worked. You might be able to use a similar technique to connect to your service and read the actual response error text.

Incardinate answered 19/7, 2011 at 23:2 Comment(2)
HttpWebRequest is so messed up. Even sockets are easier (coz they don't hide errors away from you).Misestimate
So how do I set the body?Aerometeorograph
S
6

Try this (it's VB-Code :-):

Try

Catch exp As WebException
  Dim sResponse As String = New StreamReader(exp.Response.GetResponseStream()).ReadToEnd
End Try
Satterwhite answered 25/3, 2014 at 15:2 Comment(0)
N
4

An asynchronous version of extension function:

    public static async Task<WebResponse> GetResponseAsyncNoEx(this WebRequest request)
    {
        try
        {
            return await request.GetResponseAsync();
        }
        catch(WebException ex)
        {
            return ex.Response;
        }
    }
Nankeen answered 10/2, 2015 at 16:25 Comment(0)
H
-1

This solved it for me:
https://gist.github.com/beccasaurus/929007/a8f820b153a1cfdee3d06a9c0a1d7ebfced8bb77

TL;DR:
Problem:
localhost returns expected content, remote IP alters 400 content to "Bad Request"
Solution:
Adding <httpErrors existingResponse="PassThrough"></httpErrors> to web.config/configuration/system.webServer solved this for me; now all servers (local & remote) return the exact same content (generated by me) regardless of the IP address and/or HTTP code I return.

Hertzfeld answered 22/7, 2019 at 11:21 Comment(3)
This has nothing to do with WebHttpRequest throwing exceptions on unsuccessful HTTP codes.Railing
@Railing then what is the cause? Don't just criticize without including some insight as to why you believe your comment is valid.Hertzfeld
The cause is that the WebHttpRequest class is written to throw exceptions on unsuccessful HTTP codes, which many people find unhelpful. It has nothing to do with web.config configuration which controls the server settings. The issue you are talking about (the observed HTTP payload) has nothing to do with the fact that WebHttpRequest throws exceptions on unsuccessful HTTP codes.Railing

© 2022 - 2024 — McMap. All rights reserved.