httplib2, how to set more than one cookie?

Question

As you are probably aware, more often than not, an HTTP server will send more than just a session_id cookie; however, httplib2 handles cookies with a dictionary, like this:

response, content = http.request(url, 'GET', headers=headers)

headers = {'Cookie': response['set-cookie']}

url = 'http://www.example.com/home'   
response, content = http.request(url, 'GET', headers=headers)

So, how do I set the extra cookies? If handled with a dictionary, I can't have double Cookie keys :S.

Thanks for your time.

Imprescriptible

I recommend reading the http rfc: w3.org/Protocols/rfc2616/rfc2616.html and the cookie rfc: faqs.org/rfcs/rfc2109.html. You may not think that you should reach such documents all for the sake of this little task, but actually since you're obviously doing some kind of network programming you should understand these important concepts. – Wealthy

I'll surely be reading them in a few days when I have more time, I bookmarked the links. Thanks. – Imprescriptible

Answer 1

Cookies are contained in a single HTTP header, separated by semicolons. Example:

cookie1=value1;cookie2=value2

So you'll need to build a string from the cookies sent by the server, and then set that as the Cookie header.

Edit: Actually, playing around a bit with httplib2 and re-reading your question, I'm not sure you actually need to do anything to get the functionality you want. The set-cookie value you get back from httplib2 is actually the raw Set-Cookie header sent from the server; you can just put that into the cookie header of the new response, and everything will work fine. Technically speaking you should remove some cookie attributes such as expiry, but I imagine most servers will handle that just fine.

Hoskinson

Answer 2

Yes, I just found out elsewhere about the Cookie header when making the request, but the server may send several Set-Cookie headers, with a cookie(and expiration,domain,etc attributes) per header. But with the dictionary system used in httplib2, I can't really get all the possible Set-Cookie headers sent by the server, but seemingly, just the last one.

So, any more ideas :)?

Imprescriptible

Answer 3

Doing some extra testing, with a dummy setcookie() PHP page, I generated in 3 test, the following set of headers:

Set-Cookie: chocolate=chips
Set-Cookie: milk=shape


Set-Cookie: chocolate=chips; expires=Sun, 15-Nov-2009 18:47:08 GMT; path=/; domain=thaorius.net; secure; httponly
Set-Cookie: milk=shape


Set-Cookie: chocolate=chips; expires=Sun, 15-Nov-2009 18:46:25 GMT
Set-Cookie: milk=shape

The output actually supplied by httplib2 on the set-cookie key of the array, is, respectively for each header pair, this:

chocolate=chips, milk=shape

chocolate=chips; expires=Sun, 15-Nov-2009 18:31:00 GMT; path=/; domain=thaorius.net; secure; httponly, milk=shape

chocolate=chips; expires=Sun, 15-Nov-2009 18:38:21 GMT, milk=shape

So it seems that httplib2 does deal with the problem properly, but now I'm presented with another problem. The "," in the expires attribute. As you can see, cookies get separated by a comma, but how to distinguish from that of the expires attribute.

I could split the string by commas, and then by ";", and end up with key value pairs for each cookie, nice and easy; but with the comma in expires, I can't possibly do that.

So, I'm thinking, I could use a regular expression that basically looks for "expires=letters, num&letters nums:nums:nums arbitrarychars[,|;|$]" and replaces it for something like expires=STUB, as I do not really care about the expiration time of the cookies.

So, would anyone be so kind as to give me the regex I can feed to re.sub()? I haven't really needed regex so far, thus I haven't learned them, and I really don't want to loose a few days for a single one :).

Imprescriptible

Answer 4

Try this:

pp = re.compile('(Sun|Mon|Tue|Wed|Thu|Fri|Sat),')
pp.sub('','alpha Sun, beta')
'alpha  beta'

Should