How can I programmatically remove the 2 connection limit in WebClient

Question

Those "fine" RFCs mandate from every RFC-client that they beware of not using more than 2 connections per host...

Microsoft implemented this in WebClient. I know that it can be turned off with

App.config:

<?xml version="1.0" encoding="utf-8" ?> 
<configuration> 
 <system.net> 
  <connectionManagement> 
   <add address="*" maxconnection="100" /> 
  </connectionManagement> 
 </system.net> 
</configuration> 

(found on http://social.msdn.microsoft.com/forums/en-US/netfxnetcom/thread/1f863f20-09f9-49a5-8eee-17a89b591007 )

But how can I do it programmatically?

Accordin to http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit.aspx

"Changing the DefaultConnectionLimit property has no effect on existing ServicePoint objects; it affects only ServicePoint objects that are initialized after the change. If the value of this property has not been set either directly or through configuration, the value defaults to the constant DefaultPersistentConnectionLimit."

I'd like best to configure the limit when I instanciate the WebClient, but just removing this sad limitation programmatically at the start of my programm would be fine, too.

The server I access is not a regular webserver in the internet, but under my control and in the local lan. I want to do API-calls, but I don't use webservices or remoting

Aquila

I might upvote this question if I knew why you need to violate international standards. – Groos

It's not really a standard. The RFC "recommends" that you limit clients to two connections, but it's not really a requirement. More than likely, the poster needs to download more than 2 items at one time. – Mattern

I access an API on my own server. I don't want to harm hosts in the internet. – Aquila

I've increased the connection limit to build a load test tool. It's really hard to load test with 2 measley connections. I'm sure there are lot's of non-browsing reasons to use many connections. – Farwell

BTW, the config above will affect all .Net controlled connections, not just webclient. – Farwell

Why more than two? Let's turn around the question: why I couldn't issue more than two requests to a server at the same time asynchronously? 2 is just literally a limitation. – Marvelofperu

@John Saunders. Don't worry, it's nothing devious. The reason is load testing. It is just to make sure a website will handle lots of overlapping requests well before it goes live. – Niece

@JohnRobertson: you might want to say how you know this, four years later? And, my solution would be to consider using a server OS for load test generation. – Groos

@John Saunders. Whoa there. Not trying to start a fight. – Niece

Answer 1

With some tips from here and elsewhere I managed to fix this in my application by overriding the WebClient class I was using:

class AwesomeWebClient : WebClient {
    protected override WebRequest GetWebRequest(Uri address) {
        HttpWebRequest req = (HttpWebRequest)base.GetWebRequest(address);
        req.ServicePoint.ConnectionLimit = 10;
        return (WebRequest)req;
    }
}

Irma

IMHO that setting the System.Net.ServicePointManager.DefaultConnectionLimit is a better solution, as cannot assume that the WebRequest is a HttpWebRequest, e.g., it could be a FileRequest. – Alannaalano

Answer 2

for those interested:

System.Net.ServicePointManager.DefaultConnectionLimit = x (where x is your desired number of connections)

no need for extra references

just make sure this is called BEFORE the service point is created as mentioned above in the post.

Tautonym

So could this be added to application_start in the global? so it affects all connections made? – Jeanicejeanie

How & where to add System.Net.ServicePointManager.DefaultConnectionLimit = x ? – Senn

Oddly, the code comment for DefaultConnectionLimit (navigating using F12) says its default is Int32.MaxValue. However by debug inspection it is 2 as claimed. – Vuong

Answer 3

This solution allows you to change the connection limit at any time:

private static void ConfigureServicePoint(Uri uri)
{
    var servicePoint = ServicePointManager.FindServicePoint(uri);

    // Increase the number of TCP connections from the default (2)
    servicePoint.ConnectionLimit = 40;
}

The 1st time anyone calls this FindServicePoint, a ServicePoint instance is created and a WeakReference is created to hold on to it inside the ServicePointManager. Subsequent requests to the manager for the same Uri return the same instance. If the connection isn't used after, the GC cleans it up.

Scoggins

The only problem with FindServicePoint is that it hands you back one ServicePoint but you don't know if it will be the same ServicePoint your client gets. – Needs

That's not a "problem", it's just a normal part of the job. As with all solutions, you have to find a way to test that. My way was to set the setting in the .config to "1", observe the terrible performance, and set it in code (as here), observing the improved performance. – Chivaree

The ServicePoint is lost (along with your settings) after MaxIdleTime – Wiencke

Answer 4

If you find the ServicePoint object being used by your WebClient, you can change its connection limit. HttpWebRequest objects have an accessor to retrieve the one they were constructed to use, so you could do it that way. If you're lucky, all your requests might end up sharing the same ServicePoint so you'd only have to do it once.

I don't know of any global way to change the limit. If you altered the DefaultConnectionLimit early enough in execution, you'd probably be fine.

Alternately, you could just live with the connection limit, since most server software is going to throttle you anyway. :)

Paravane

This server will not throttle me (in fact, it will, but in a different way) as it is completely under my control – Aquila

A server might throttle with a lot of connections, but I haven't experienced that even with a small server (hosted in a limited VM). The limit 2 on the client side held me back on the other hand. Increasing the limit liberated the situation. – Marvelofperu

I also doubt that any of today's browser would obey to the HTTP 1.1 RFC's limit of 2. – Marvelofperu

Answer 5

We have a situation regarding the above piece of configuration in App.Config

In order for this to be valid in a CONSOLE Application, we added the System.Configuration reference dll. Without the reference, the above was useless.

Cle