System.Net.WebException: The operation has timed out

Question

I have a big problem: I need to send 200 objects at once and avoid timeouts.

while (true)
{

    NameValueCollection data = new NameValueCollection();
    data.Add("mode", nat);

    using (var client = new WebClient())
    {
        byte[] response = client.UploadValues(serverA, data);
        responseData = Encoding.ASCII.GetString(response);

        string[] split = Javab.Split(new[] { '!' },  StringSplitOptions.RemoveEmptyEntries);
        string command = split[0];
        string server = split[1];
        string requestCountStr = split[2];

        switch (command)
        {
            case "check":
                int requestCount = Convert.ToInt32(requestCountStr);

                for (int i = 0; i < requestCount; i++)
                {
                    Uri myUri = new Uri(server);
                    WebRequest request = WebRequest.Create(myUri);
                    request.Timeout = 200000;
                    WebResponse myWebResponse = request.GetResponse();
                }
                break;
        }
    }    
}

This produces the error:

Unhandled Exception: System.Net.WebException: The operation has timed out  
at System.Net.HttpWebRequest.GetResponse()  
at vir_fu.Program.Main(String[] args)

The requestCount loop works fine outside my base code but when I add it to my project I get this error. I have tried setting request.Timeout = 200; but it didn't help.

Gery

BTW, when you post code, try to post real code. Your code would not compile: the WebRequest constructor is protected. It would also be convenient if instead of "...", you used "// ...". – Landside

Answer 1

It means what it says. The operation took too long to complete.

BTW, look at WebRequest.Timeout and you'll see that you've set your timeout for 1/5 second.

Landside

A valuable guide.. Thank you!! – Deepsea

Answer 2

Close/dispose your WebResponse object.

Martinez

closing/disposing doesn't help – Entertain

Answer 3

I'm not sure about your first code sample where you use WebClient.UploadValues, it's not really enough to go on, could you paste more of your surrounding code? Regarding your WebRequest code, there are two things at play here:

1. You're only requesting the headers of the response**, you never read the body of the response by opening and reading (to its end) the ResponseStream. Because of this, the WebRequest client helpfully leaves the connection open, expecting you to request the body at any moment. Until you either read the response body to completion (which will automatically close the stream for you), clean up and close the stream (or the WebRequest instance) or wait for the GC to do its thing, your connection will remain open.

2. You have a default maximum amount of active connections to the same host of 2. This means you use up your first two connections and then never dispose of them so your client isn't given the chance to complete the next request before it reaches its timeout (which is milliseconds, btw, so you've set it to 0.2 seconds - the default should be fine).

If you don't want the body of the response (or you've just uploaded or POSTed something and aren't expecting a response), simply close the stream, or the client, which will close the stream for you.

The easiest way to fix this is to make sure you use using blocks on disposable objects:

for (int i = 0; i < ops1; i++)
{
    Uri myUri = new Uri(site);
    WebRequest myWebRequest = WebRequest.Create(myUri);
    //myWebRequest.Timeout = 200;
    using (WebResponse myWebResponse = myWebRequest.GetResponse())
    {
        // Do what you want with myWebResponse.Headers.
    } // Your response will be disposed of here
}

Another solution is to allow 200 concurrent connections to the same host. However, unless you're planning to multi-thread this operation so you'd need multiple, concurrent connections, this won't really help you:

 ServicePointManager.DefaultConnectionLimit = 200;

When you're getting timeouts within code, the best thing to do is try to recreate that timeout outside of your code. If you can't, the problem probably lies with your code. I usually use cURL for that, or just a web browser if it's a simple GET request.

** In reality, you're actually requesting the first chunk of data from the response, which contains the HTTP headers, and also the start of the body. This is why it's possible to read HTTP header info (such as Content-Encoding, Set-Cookie etc) before reading from the output stream. As you read the stream, further data is retrieved from the server. WebRequest's connection to the server is kept open until you reach the end of this stream (effectively closing it as it's not seekable), manually close it yourself or it is disposed of. There's more about this here.

Noggin

@Matthew: are you sure about sending HEAD? I don't think I've ever seen that happen. – Landside

Sorry, I don't mean HEAD (HEAD's where you request only the headers), I mean the initial response contains only the headers (which is what you'd normally get if you just sent a HEAD request). I've edited to fix my mistake. – Noggin

@Matthew: again, what makes you think the response only contains headers? Are you suggesting there's another packet exchange between the client and server at the time that the GetRequestStream is called? – Landside

hi matthew B , thank you for responds ,ok my code do this : 1 - start while(true)//work right 1 - post data to serverA//work right 2 - receive code//work right 3 - use switch(code)//work right 4 - send GET request to serverB //problem is here 5 - sleep for 2ming 6 - end of while(); – Gery

@John I've just done some testing with MS Network Monitor to see what happens while WebRequest does it stuff. WebRequest does exchange packets with the server when the stream is read, it seems. However, for small files, GetResponse downloads the entire body, but keeps the connection active. For anything bigger, it only requests more of the file as the user begins to read the stream. I've updated my answer with more info. – Noggin

Perhaps instead of // Do what you want with the HEAD it should read // Do what you want with myWebResponse.Headers. That may be less confusing to readers. – Insect

+1 on the DefaultConnectionLimit. As this was timing out before it even attempted the connection, it rules out any TCP/networkign timeouts. – Refract

where to set the servicepointmanager.defaultconnection ? inside using block? please suggest.. – Pegeen

Answer 4

proxy issue can cause this. IIS webconfig put this in

<defaultProxy useDefaultCredentials="true" enabled="true">
          <proxy usesystemdefault="True" />
        </defaultProxy>

Largescale

Answer 5

I remember I had the same problem a while back using WCF due the quantity of the data I was passing. I remember I changed timeouts everywhere but the problem persisted. What I finally did was open the connection as stream request, I needed to change the client and the server side, but it work that way. Since it was a stream connection, the server kept reading until the stream ended.

Aberrant

Answer 6

I encountered the same error than adding

Task.Delay(2000);

in each request solved the problem

Ranique

While this type of solution might actually work, I would not recommend it. This artificially slows down the process by forcing it to take 2 seconds per iteration. with 200 items to process, that means it will take 6+ minutes to complete, and doesn't really resolve the root issue of the timeout. I would focus on controlling the stream as suggested above, keeping the connections open only as necessary. this should allow your code to perform at a higher performance level. – Carry