Can't get any cookies with C# HttpClient

Question

I'm trying to get cookies on the Spotify login page with C# and the HttpClient class. However, the CookieContainer is always empty when I know cookies are being set. I'm not sending any headers, but it should still give me the cookie(s) because when I send a GET request without any headers with python (requests module) I get the csrf token. Here's my code:

using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Collections;
using System.Web;

class Program
{
    static void Main()
    {
        Task t = new Task(MakeRequest);
        t.Start();
        Console.WriteLine("Getting cookies!");
        Console.ReadLine();
    }

    static async void MakeRequest()
    {
        CookieContainer cookies = new CookieContainer();
        HttpClientHandler handler = new HttpClientHandler();

        handler.CookieContainer = cookies;
        Uri uri = new Uri("https://accounts.spotify.com/en/login/?_locale=en-US&continue=https:%2F%2Fwww.spotify.com%2Fus%2Faccount%2Foverview%2F");
        HttpClient client = new HttpClient(handler);
        var response = await client.GetAsync(uri);
        string res = await response.Content.ReadAsStringAsync();
        Console.WriteLine(cookies.Count);
        foreach (var cookie in cookies.GetCookies(uri)) {
            Console.WriteLine(cookie.ToString());
        }
    }
}

It seems pretty simple to me, but the program always says there's 0 cookies. Anyone know what's going on?

Vestiary

What makes you think there are cookies being sent back? – Benedictus

@Benedictus I stated that I tested the same program in python and when I tried to get the cookies I got the csrf token. – Vestiary

@CrispApples did you enable cookies on the handler handler.UseCookies? – Michaeu

This may be a dupe #14681644 – Michaeu

I tested what you have and can reproduce the problem. How ever when I call other URLs withing the root domain you have listed, cookies are being returned in the container. – Michaeu

UseCookies = true is the default setting. I suggest using a static HttpClient. Possibly a Lazy<HttpClient> httpClient = new Lazy<HttpClient>(() => { HttpClientHandler handler = new HttpClientHandler { (set a new CookieContainer here or use a static one -> to store the Cookies of all "sessions") } etc }. IMO, it works much better in all situations. – Could

If using FW 4.7.2, the HttpClientHandler also supports the SslProtocols property, which allows to set the SSL protocols directly through the handler (besides the ServerCertificateCustomValidationCallback). Required, when a connection uses Https, which might also be the problem here. – Could

@Could I’ll be sure to try both of those! – Vestiary

Answer 1

I tried to write the response headers to the console with Console.WriteLine(response.Headers) and a Set-Cookie header with the csrf token was printed to the console. So it seems that HttpClient doesn’t count cookies in this header as actual cookies, thus not adding these said cookies to the CookieContainer.

Vestiary

I am encountering the same issue but with a different URL it does detect cookies being set via headers. – Waldenburg

Answer 2

You need to enable the use of cookies using HttpClientHandler.UseCookies Property

public bool UseCookies { get; set; }

Gets or sets a value that indicates whether the handler uses the CookieContainer property to store server cookies and uses these cookies when sending requests.

//...

CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
handler.UseCookies = true; //<-- Enable the use of cookies.

//...

Michaeu

Just tried this, same result... No cookies were returned. Do you think it’s because HttpClient doesn’t account for cookies in the Set-Cookie response header? Take a look at my answer. – Vestiary

@CrispApples I tested this going to https://spotify.com and got back cookies. same Set-Cookie header with multiple entries. container seems empty for the original URL you provided – Michaeu

@CrispApples I also tested it with multiple URL within the spotify domain, the cookie container is empty for any calls with accounts.spotify.com even with the Set-Cookie header present – Michaeu

That’s weird. I got cookies back on spotify.com too. Maybe accounts.spotify.com’s Set-Cookie header isn’t formatted right or something? Either way, I’d say this question is solved because I can just extract the csrf token from the Set-Cookie response header even if HttpClient doesn’t pick it up itself. – Vestiary

Even enabling UseCookies to true, did not receive cookies – Mosquito

Answer 3

Make sure you are decompression the response automatically. See my answer here: https://mcmap.net/q/393016/-httpclient-not-storing-cookies-in-cookiecontainer

An example of what I use:

var clientHandler = new HttpClientHandler {
    AllowAutoRedirect = true,
    UseCookies = true,
    CookieContainer = cookieContainer,
    AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate, // <--
};

Porch