Why does WebRequest timeout always on the first request, but never on any subsequent ones

Question

Having an issue, where calling WebRequest.GetResponse() hangs and times out on the first call, but after the first call, everything works fine.

        try {
            WebRequest myHttpWebRequest = WebRequest.Create(@"http://192.168.x.x/");
            // Sends the HttpWebRequest and waits for the response.         
            myHttpWebRequest.Timeout = 1000;
            WebResponse myHttpWebResponse = myHttpWebRequest.GetResponse();
        } catch(Exception e) {
            Console.WriteLine("Failure 1");
        }
        try {
            WebRequest myHttpWebRequest = WebRequest.Create(@"http://192.168.x.x/");
            // Sends the HttpWebRequest and waits for the response.         
            myHttpWebRequest.Timeout = 1000;
            WebResponse myHttpWebResponse = myHttpWebRequest.GetResponse(); 
        } catch(Exception e) {
            Console.WriteLine("Failure 2");
        }
        try {
            WebRequest myHttpWebRequest = WebRequest.Create(@"http://192.168.x.x/");
            // Sends the HttpWebRequest and waits for the response.         
            myHttpWebRequest.Timeout = 1000;
            WebResponse myHttpWebResponse = myHttpWebRequest.GetResponse(); 
        } catch(Exception e) {
            Console.WriteLine("Failure 3");
        }

using this code in a console application, I always receive a Failure 1. Running under the debugger or not. I've done a 1000 loop, and it always fails on the first one, never any other ones. In fact, reading the logs of the web server, it actually never receives the first request. Am I missing something here?

Reseta

Have you ever got the solution for this issue ' – Pantalets

Yes, it was the result of having a VPN running. Turning off the VPN or VPN like software fixed the problem. – Reseta

Answer 1

EDIT: I've realised the answer below would fit with the exact opposite situation, where the first request works but the others don't. However, it's still important - you really should be disposing of your responses. It would also be useful if when you report the error, you also report the exception message...

To work out what's going on here, you should really use something like WireShark so you can see whether the problem is that the request is being made but not responded to, or whether it's not even being made.

I wonder whether the problem is actually that it's resolving a proxy, or something like that... and there's just about enough time to resolve it before the second request times out. Try increasing the timeouts. Again, this should be visible via WireShark.

---

You're not disposing of the web response, so the connection pool for the second request is going to time out waiting to get that connection back.

Put the WebResponse part in a using statement and you'll probably find it all works fine:

using (WebResponse myHttpWebResponse = myHttpWebRequest.GetResponse())
{
}

That's assuming you'd actually do something with the response, of course. Otherwise you could just write:

myHttpWebRequest.GetResponse().Dispose();

:)

Alyose

But its not the second request that i'm getting the timeout, in fact, the second one works perfectly fine. The first one is timing out. But, I did make the change like you said, and did the using (this is just a test not a real app, yet), and it made no difference. The first request is still timing out. – Reseta

@Jeremy: Yes, I just noticed that. Is the web site up before the first request? – Alyose

I did report the exception, its a timeout – Reseta

Yes the internal web server is always up. In fact, i'm doing a tail on the access log, to see the requests come in, the first one NEVER makes it to the server. – Reseta

@Jeremy: I mean including it in your console output. See my edit though for a suggested way forward - you really need to look at what's happening on the network. – Alyose

Already done that though, no traffic is sent to the server for the first request, but the second and third come through just fine. Also, dns is not an issue since its by ip address. Stepping through the code in the debugger, and watching traffic and access logs, produces nothing. – Reseta

@Jeremy: DNS may not be an issue, but proxy resolution might be. Look at all network traffic from this process. What happens if you increase the timeout to several seconds? – Alyose

You were correct, it was the result of a VPN actually. A C or Delphi application handles this correctly with no timeout. So does internet explorer and other web browsers. Is there a bug in .NET where it doesn't use the default gateway and just blindly tries VPNs? – Reseta

@Jeremy: I would expect .NET to just use the OS-level network layer. Sounds odd to me... – Alyose

Yeah, thanks for the help. I have the VPNs set to not use the VPN default gatewau for standard network traffic, but use mine. Which works just fine for applications. Strangely enough though, this is an internal IP address, times out if i use www.google.com also, so it shouldn't be using an internet gateway. – Reseta

Using Statement works fine.it's give me the network exception little bit quickly than the waiting for some time out exception. – Singley

Answer 2

May be a little late, but I had exactly the same effect. The reason finally was, that there has been no default gateway in the network. The solution was to optianally set the request.Proxy = null.

var request = WebRequest.Create(UriString);
request.Timeout = Timeout;
if (_disableProxy)
{
    request.Proxy = null;
}
if (request is HttpWebRequest)
{
    var response = (HttpWebResponse)request.GetResponse();
    responseStream = response.GetResponseStream();

}
if (request is FtpWebRequest)
{
    var response = (FtpWebResponse)request.GetResponse();
    responseStream = response.GetResponseStream();
}
else if (request is FileWebRequest)
{
    var response = (FileWebResponse)request.GetResponse();
    responseStream = response.GetResponseStream();
}

Hope this helps.

Shrewmouse

Answer 3

Late response here but hopefully it might help someone.

I found my issue related to running the HTTP request in a new thread. For some reason, the first request always timed out and after that, everything was fine. I could see that the HTTP request was not hitting the server which means something internal to .NET was blocking it (for the record, ServicePointManager.DefaultConnectionLimit was not the issue).

The solution for me was to put the request in a Task instead of Thread.

(Using .NET Framework 4.x)

Terr

I'm having the same issue now. My page fetch is already in a thread. So the question is why is it behaving like this because I don't fancy refactoring my code to do the request in a task. – Hidrosis

Answer 4

You can get a behavior very similar to this if you haven't flushed\closed the RequestStream prior to asking for the Response. This behavior seems to exist on .NET 3.5 but has been addressed in .NET Framework 4.5. I noticed the problem when switching frameworks - code (w/o the close) that worked in 4.5 stopped working when compiled against 3.5. Maybe try explicitly getting the RequestStream and closing it as a work-around.

Jamesy

That's pretty much what John explained, and I don't see closing a stream as a "workaround" but more a "must do". – Squire

Answer 5

I have encountered the same issue, in my case I have increased the timeout value of the WebRequest object and it worked!

webRequest.Timeout = int.Parse(60000);

(I have set the timeout property to 60 Seconds).

Chaw

Sounds more like a timeout issue.... i had timeout set to 3600000 and i stopped it after 12 mins... – Reseta

3600000 is pretty huge, – Chaw

I was testing at the time – Reseta

Well, best of luck then! – Chaw

I figured it out... had to do with a vpn... stopped the vpn and everything worked perfectly on the first try – Reseta

Fyi... i found this out the hard way.. timeout is the time for the entire request... not just of inactivity... so if you have a large post on a slow connection.. the timeout might happen even if it is actively sending the data – Reseta