How to parse HttpWebResponse.Headers.Keys for a Set-Cookie session id returned

Question

I'm trying to create an HttpWebRequest/HttpWebResponse session with an ASP.NET website to later parse an HTML form through url params (this part I know how to do), but I do not understand how to parse and set a cookie such as the session id. In Fiddler, it shows that the ASP.NET Session ID is returned through Set-Cookie in the response to the request to the / path of the url, but how can I extract this session id and set it as a cookie for the next HttpWebRequest? I understand that this Set-Cookie header would be found in HttpWebResponse.Headers.Keys, but is there a direct path to parsing it? Thanks!

Melanite

Answer 1

The .NET framework will manage cookies for you. You don't have to concern yourself with parsing the cookie information out of the headers or adding a cookie header to your requests.

To store and send your session ID, use the Cookie and CookieContainer classes to store them and then make sure you send your cookies with every request.

The following example shows how to do this. The CookieContainer, 'cookieJar' can be shared across multiple domains and requests. Once you add it to a request object, the reference to it will also be added to the response object when the response is returned.

CookieContainer cookieJar = new CookieContainer();

var request = (HttpWebRequest)HttpWebRequest.Create("http://www.google.com");
request.CookieContainer = cookieJar;

var response = request.GetResponse();

foreach (Cookie c in cookieJar.GetCookies(request.RequestUri))
{
    Console.WriteLine("Cookie['" + c.Name + "']: " + c.Value);
}

The output of this code will be:

Cookie['PREF']: ID=59e9a22a8cac2435:TM=1246226400:LM=1246226400:S=tvWTnbBhK4N7Tlpu

Gracegraceful

+1 Thanks for that much appreciated help! I wasn't looking forward to manually parsing HTTP headers :) – Johannejohannes

That's an unexpected behavior... response.Cookies shouldn't be empty when CookieContainer is not defined. – Weakling

@DaviFiamenghi It is, however, the documented behavior. Microsoft expects you to have a single cookie management object to share across web requests/responses. If you don't do this, you will need to manually parse the response's headers for cookies. – Gracegraceful

@DanHerbert Thats nice. But I still think that it would be more intuitive if an InvalidOperationException be thrown when you try to access the "Cookies" property of the response without a Container defined. When I saw 0 cookies, I thought that there was 0 cookies in the response. This behavior would be just like when you try to read a stream that is disposed or something like... and I believe that if an explanatory exception was thrown, it would prevent people from asking on SO or rolling your own cookie parsing logic. – Weakling

Answer 2

The answer from Dan Herbert helped me really. I appreciate your help.

Just want to post my usage - hope it helps some one at some point of time. My requirement is that I need to send back cookies from first http post response to second http post request.

1st:

CookieContainer cookieJar = new CookieContainer();
request.CookieContainer = cookieJar;
....

CookieCollection setCookies = cookieJar.GetCookies(request.RequestUri);

2nd:

CookieContainer cc = new CookieContainer();
cc.Add(setCookies);    
request.CookieContainer = cc;

Timbuktu

Answer 3

I have the same problem (with amazon) I use the following regexp:

string regexp = "(?<name>[^=]+)=(?<val>[^;]+)[^,]+,?";);
MatchCollection myMatchCollection = Regex.Matches(cookiesStr, regexp);
foreach (Match myMatch in myMatchCollection)
                {
                    string cookieName = myMatch.Groups["name"].ToString();
                    string cookieVal = myMatch.Groups["val"].ToString();
                    Cookie cookie = new Cookie(cookieName, cookieVal);
                    cookies.Add(cookie);                
                }

Note that I only care about the cookie name/value...

good luck Elia

Provenience

This is the exact type of solution I am looking for (WebBrowser object only provides a string for the cookie). However, your code doesn't compile. – Arrow

Answer 4

hum I may be wrong but from what I am observing lately

Cookies from a first response, don't include the 'set cookie' as cookies that come in the header (for example some session id...) in the case of a 302 (redirect) status

If the autofollowredirect is set to true, then the set cookie are processed, and the subsequent request which is done automatically, will include those cookies defined by set cookie on the first call

If autofollowredirect is set to false then the first request doesn't get the cookies positionned by the set cookie, and I guess and this is also my queston if anyone know, that the only way to subsequently have those cookies in next request, is parse the set cookies ?

Sinclare

I had issues with a 302 redirect and set the AllowAutoRedirect = false. This gave me a "this page has moved" response, but then I pas the cookies along and redirect to the page that I wanted to go to in the first place. – Baton

According to RFC 6265 User agents MAY ignore Set-Cookie headers contained in responses with 100-level status codes but MUST process Set-Cookie headers contained in other responses (including responses with 400- and 500-level status codes). – Granddaughter