Using CookieContainer with WebClient class

Question

I've previously used a CookieContainer with HttpWebRequest and HttpWebResponse sessions, but now, I want to use it with a WebClient. As far as I understand, there is no built-in method like there is for HttpWebRequests (request.CookieContainer). How can I collect cookies from a WebClient in a CookieContainer?

I googled for this and found the following sample:

public class CookieAwareWebClient : WebClient
{
    private readonly CookieContainer m_container = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest request = base.GetWebRequest(address);
        HttpWebRequest webRequest = request as HttpWebRequest;
        if (webRequest != null)
        {
            webRequest.CookieContainer = m_container;
        }
        return request;
    }
}

Is this the best way to do it?

Ellmyer

From my point of view m_container is never set!? Isnt it always empty? – Tetryl

I believe that the HttpWebRequest class modifies the m_container class using its internal field CookieContainer as needed. – Retractile

This is all you need! The cookies from the responses will get added to the container automatically. – Decrease

Answer 1

Yes. IMHO, overriding GetWebRequest() is the best solution to WebClient's limited functionalty. Before I knew about this option, I wrote lots of really painful code at the HttpWebRequest layer because WebClient almost, but not quite, did what I needed. Derivation is much easier.

Another option is to use the regular WebClient class, but manually populate the Cookie header before making the request and then pull out the Set-Cookies header on the response. There are helper methods on the CookieContainer class which make creating and parsing these headers easier: CookieContainer.SetCookies() and CookieContainer.GetCookieHeader(), respectively.

I prefer the former approach since it's easier for the caller and requires less repetitive code than the second option. Also, the derivation approach works the same way for multiple extensibility scenarios (e.g. cookies, proxies, etc.).

Vorster

Answer 2

 WebClient wb = new WebClient();
 wb.Headers.Add(HttpRequestHeader.Cookie, "somecookie");

From Comments

How do you format the name and value of the cookie in place of "somecookie" ?

wb.Headers.Add(HttpRequestHeader.Cookie, "cookiename=cookievalue"); 

For multiple cookies:

wb.Headers.Add(HttpRequestHeader.Cookie, 
              "cookiename1=cookievalue1;" +
              "cookiename2=cookievalue2");

Straightforward

How do you format the name and value of the cookie in place of "somecookie" ? – Agglutinate

@Neil N: wb.Headers.Add(HttpRequestHeader.Cookie, "cookiename=cookievalue"); For multiple cookies: wb.Headers.Add(HttpRequestHeader.Cookie, "cookiename1=cookievalue1; cookiename2=cookievalue2"); – Keister

This was the most easy way for me, to transport a cookie from Anglesharp to WebClient for a download-link :-) Thank you! – Vedetta

Answer 3

This one is just extension of article you found.

public class WebClientEx : WebClient
{
    public WebClientEx(CookieContainer container)
    {
        this.container = container;
    }

    public CookieContainer CookieContainer
        {
            get { return container; }
            set { container= value; }
        }

    private CookieContainer container = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest r = base.GetWebRequest(address);
        var request = r as HttpWebRequest;
        if (request != null)
        {
            request.CookieContainer = container;
        }
        return r;
    }

    protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
    {
        WebResponse response = base.GetWebResponse(request, result);
        ReadCookies(response);
        return response;
    }

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        WebResponse response = base.GetWebResponse(request);
        ReadCookies(response);
        return response;
    }

    private void ReadCookies(WebResponse r)
    {
        var response = r as HttpWebResponse;
        if (response != null)
        {
            CookieCollection cookies = response.Cookies;
            container.Add(cookies);
        }
    }
}

Maniacal

This worked well @Pavel, though you could improve this answer by showing how to use the features of the class, especially setting and getting the cookies on it. – Holster

Thx for extension. For use it I add public CookieContainer CookieContainer { get { return _container; } set { _container = value; } } – Willywillynilly

@IgorShubin you have to remove the readonly modifier of the container field, otherwise you cannot set it in the property. I modified the code. – Rubberneck

@Rubberneck - agreed. However, in you edit, your default constructor will leave CookieContainer null, whereas in the original answer it's allocated by default. – Aharon

Shouldn't you check Set-Cookie response header in ReadCookies ? – Iconoduly

You don't actually need the GetWebResponse and ReadCookies, since the cookies will get added to the container automatically. – Decrease

@achilles httpWebResponse.Cookies == Set-Cookie Response Header. here:msdn.microsoft.com/en-us/library/… . – Tabina

Can you see this? #48278833 – Noncompliance

Answer 4

The HttpWebRequest modifies the CookieContainer assigned to it. There is no need to process returned cookies. Simply assign your cookie container to every web request.

public class CookieAwareWebClient : WebClient
{
    public CookieContainer CookieContainer { get; set; } = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri uri)
    {
        WebRequest request = base.GetWebRequest(uri);
        if (request is HttpWebRequest)
        {
            (request as HttpWebRequest).CookieContainer = CookieContainer;
        }
        return request;
    }
}

Salep

Answer 5

I think there's cleaner way where you don't have to create a new webclient (and it'll work with 3rd party libraries as well)

internal static class MyWebRequestCreator
{
    private static IWebRequestCreate myCreator;

    public static IWebRequestCreate MyHttp
    {
        get
        {
            if (myCreator == null)
            {
                myCreator = new MyHttpRequestCreator();
            }
            return myCreator;
        }
    }

    private class MyHttpRequestCreator : IWebRequestCreate
    {
        public WebRequest Create(Uri uri)
        {
            var req = System.Net.WebRequest.CreateHttp(uri);
            req.CookieContainer = new CookieContainer();
            return req;
        }
    }
}

Now all you have to do is opt in for which domains you want to use this:

    WebRequest.RegisterPrefix("http://example.com/", MyWebRequestCreator.MyHttp);

That means ANY webrequest that goes to example.com will now use your custom webrequest creator, including the standard webclient. This approach means you don't have to touch all you code. You just call the register prefix once and be done with it. You can also register for "http" prefix to opt in for everything everywhere.

Knorr

I'm not sure about the last couple of sentences; the docs say: "The HttpWebRequest class is registered to service requests for HTTP and HTTPS schemes by default. Attempts to register a different WebRequest descendant for these schemes will fail." – Couteau