Binding an IP address just works for the first time
Asked Answered
K

4

12

I want to make a web request from one of available IP addresses on server so I use this class:

public class UseIP
{
    public string IP { get; private set; }

    public UseIP(string IP)
    {
        this.IP = IP;
    }

    public HttpWebRequest CreateWebRequest(Uri uri)
    {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.BindIPEndPointDelegate = new BindIPEndPoint(Bind);
        return WebRequest.Create(uri) as HttpWebRequest;
    }

    private IPEndPoint Bind(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
    {
        IPAddress address = IPAddress.Parse(this.IP);
        return new IPEndPoint(address, 0);
    }
}

Then:

UseIP useIP = new UseIP("Valid IP address here...");
Uri uri = new Uri("http://ip.nefsc.noaa.gov");
HttpWebRequest request = useIP.CreateWebRequest(uri);
// Then make the request with the specified IP address

But the solution just works the first time!

Keese answered 24/5, 2011 at 18:1 Comment(7)
Yes, I want to rapidly change my IP address. What approach should I go?Keese
Are you getting any exceptions?Omnidirectional
You could try to make the Binding only the first time and keep it in a static or instance variable?Hooper
@alexD: No. Just the first bound IP works. Subsequent instances of UseIP class will use the same IP address.Keese
The bounty will be awarded for a solution to "changing IP address of web requests frequently". Something like UseIP class which works.Keese
I am not sure what is wrong. Running the code sample in a test project (translated to VB.NET) and it worked as expected. The IP address returned by the CreateWebRequest method has the updated IP address. I didn't post my sample as an answer as I didn't seem to have done anything special.Rodie
@Frazell: Try changing the IP address without closing the application. On subsequent changes the application uses the first IP instead of IP you mentioned.Keese
A
18

A theory:

HttpWebRequest relies on an underlying ServicePoint. The ServicePoint represents the actual connection to the URL. Much in the same way your browser keeps a connection to a URL open between requests and reuses that connection (to eliminate the overhead of opening and closing the connection with each request), ServicePoint performs the same function for HttpWebRequest.

I think that the BindIPEndPointDelegate that you are setting for the ServicePoint is not being called on each use of HttpWebRequest because the ServicePoint is reusing the connection. If you could force the connection to close, then the next call to that URL should cause the ServicePoint to need to call BindIPEndPointDelegate again.

Unfortunately, it doesn't appear that the ServicePoint interface gives you the ability to directly force a connection to close.

Two solutions (each with slightly different results)

1) For each request, set HttpWebRequest.KeepAlive = false. In my test, this caused the Bind delegate to get called one-for-one with each request.

2) Set the ServicePoint ConnectionLeaseTimeout property to zero or some small value. This will have the effect of periodically forcing the Bind delegate to be called (not one-for-one with each request).

From the documentation:

You can use this property to ensure that a ServicePoint object's active connections do not remain open indefinitely. This property is intended for scenarios where connections should be dropped and reestablished periodically, such as load balancing scenarios.

By default, when KeepAlive is true for a request, the MaxIdleTime property sets the time-out for closing ServicePoint connections due to inactivity. If the ServicePoint has active connections, MaxIdleTime has no effect and the connections remain open indefinitely.

When the ConnectionLeaseTimeout property is set to a value other than -1, and after the specified time elapses, an active ServicePoint connection is closed after servicing a request by setting KeepAlive to false in that request.

Setting this value affects all connections managed by the ServicePoint object.

public class UseIP
{
    public string IP { get; private set; }

    public UseIP(string IP)
    {
        this.IP = IP;
    }

    public HttpWebRequest CreateWebRequest(Uri uri)
    {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.BindIPEndPointDelegate = (servicePoint, remoteEndPoint, retryCount) =>
        {
            IPAddress address = IPAddress.Parse(this.IP);
            return new IPEndPoint(address, 0);
        };

        //Will cause bind to be called periodically
        servicePoint.ConnectionLeaseTimeout = 0;

        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
        //will cause bind to be called for each request (as long as the consumer of the request doesn't set it back to true!
        req.KeepAlive = false;

        return req;
    }
}

The following (basic) test results in the Bind delegate getting called for each request:

static void Main(string[] args)
    {
        //Note, I don't have a multihomed machine, so I'm not using the IP in my test implementation.  The bind delegate increments a counter and returns IPAddress.Any.
        UseIP ip = new UseIP("111.111.111.111");

        for (int i = 0; i < 100; ++i)
        {
            HttpWebRequest req = ip.CreateWebRequest(new Uri("http://www.yahoo.com"));
            using (WebResponse response = req.GetResponse())
            {
            }
        }

        Console.WriteLine(string.Format("Req: {0}", UseIP.RequestCount));
        Console.WriteLine(string.Format("Bind: {0}", UseIP.BindCount));
    }
Antepast answered 28/5, 2011 at 23:33 Comment(6)
Summary - if you want each request to come from a new IP address, make sure HttpWebRequest.KeepAlive is false for each request. Performance will suffer because you are opening and closing connections with each request. Use ConnectionLeaseTimeout if you want to occassionally force a new IP address to be used for a given URI.Antepast
@Joe: I have tested setting KeepAlive of HttpWebRequest to false. Some websites refuse to service such a client.Keese
Ok - if that is the case, you have two options: Use ConnectionLeaseTimeout=0. You will occasionally reuse IP addresses, but about 60% (from my test) of your requests will call the Bind delegate. If that is not acceptable, then using HttpWebRequest wont' work for you. You need to write your own version of a web client that sends a KeepAlive header but closes the connection after the request. You can do this using the System.Net.Socket class.Antepast
@Joe: What about cheating the remote server. Using HttpWebServer with no KeepAlive but set the http header manually?Keese
@Xaqron: When I tried this (by using req.Headers.Add("Connection", "Keep-Alive");), it throws an exception saying you must use the appropriate property instead. I don't think it's possible to set this particular header manually. I find it odd that some servers would reject requests with no Keep-Alive header - as far as the HTTP protocol is concerned they shouldn't be counting on it. It takes me back to the suggestion that to get the behavior you need you are probably going to have to write your own implementation of HttpWebRequest.Antepast
@Joe: Using reflection there's a hack to set those headers.Keese
B
1

Problem may be with the delegate getting reset on each new request. Try below:

//servicePoint.BindIPEndPointDelegate = null; // Clears all delegates first, for testing
servicePoint.BindIPEndPointDelegate += delegate
    {
        var address = IPAddress.Parse(this.IP);
        return new IPEndPoint(address, 0);
    };

Also as far as I know, the endpoints are cached so even clearing the delegate may not work in some cases and they may get reset regardless. You may unload/reload the app domain as the worst case scenario.

Burlie answered 28/5, 2011 at 23:57 Comment(0)
P
0

I have changed your example a little and make it work on my machine:

public HttpWebRequest CreateWebRequest(Uri uri)
{
    HttpWebRequest wr = WebRequest.Create(uri) as HttpWebRequest;
    wr.ServicePoint.BindIPEndPointDelegate = new BindIPEndPoint(Bind);
    return wr;
}

I did that because:

  • I think the call to FindServicePoint actually does the request using the "default" ip, without even calling the binding delegate, to the URI you have specified. In my machine, at least, the BindIPEndPointDelegate was not called in the way you have presented (I know the request was made because I didn't set the Proxy and got a proxy authentication error);
  • In the documentation of ServicePointManager, it states that "If there is an existing ServicePoint object for that host and scheme, the ServicePointManager object returns the existing ServicePoint object; otherwise, the ServicePointManager object creates a new ServicePoint object" witch would probably return always the same ServicePoint if the URI was the same (maybe explaining why the subsequent calls is happening in the same EndPoint).
  • In this way we can be sure that, even when the URI has already been requested, it will use the desired IP instead of using some previous "caching" of ServicePointManager.
Powys answered 27/5, 2011 at 14:32 Comment(2)
I have tried this before. If you change the IP address rapidly you will find it doesn't work. The problem is delegate should be static so you cannot connect the same Uri from different IP addresses at the same time.Keese
re: probably always return the same ServicePoint for the same Uri. True!. The oddity is that it returns the same ServicePoint for any Uri that addresses the same remote IPAddress. E.g. http :// 1.2.3.4/FirstTarget and http :// 1.2.3.4/SecondTarget return the same ServicePoint.Asteria
A
0

I like this new class UseIP.

There is a point at Specify the outgoing IP Address to use with WCF client about protecting yourself from IPv4/IPv6 differences.

The only thing that would need to change is the Bind method to be like this:

private IPEndPoint Bind(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
{
    if ((null != IP) && (IP.AddressFamily == remoteEndPoint.AddressFamily))
        return new IPEndPoint(this.IP, 0);
    if (AddressFamily.InterNetworkV6 == remoteEndPoint.AddressFamily)
        return new IPEndPoint(IPAddress.IPv6Any, 0);
    return new IPEndPoint(IPAddress.Any, 0);
}

re: the Bind method being called multiple times.

What works for me is to remove any delegate link before I add it.

ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
servicePoint.BindIPEndPointDelegate -= this.Bind;   // avoid duplicate calls to Bind
servicePoint.BindIPEndPointDelegate += this.Bind;

I also like the idea of caching the UseIP objects. So I added this static method to the UseIP class.

private static Dictionary<IPAddress, UseIP> _eachNIC = new Dictionary<IPAddress, UseIP>();
public static UseIP ForNIC(IPAddress nic)
{
    lock (_eachNIC)
    {
        UseIP useIP = null;
        if (!_eachNIC.TryGetValue(nic, out useIP))
        {
            useIP = new UseIP(nic);
            _eachNIC.Add(nic, useIP);
        }
        return useIP;
    }
}
Asteria answered 28/1, 2013 at 1:52 Comment(1)
Oops. Sorry. I changed the Type of the property IP to be IPAddress so I wouldn't have to parse it every time. I forgot to mention that.Asteria

© 2022 - 2024 — McMap. All rights reserved.