HttpClient with .Net Core 2.1 hangs

Question

Given the following .Net Core 2.1 Console App...

using System;
using System.Diagnostics;
using System.Net.Http;
using System.Net.Http.Headers;

namespace TestHttpClient
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (var httpClient = new HttpClient())
                {
                    httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));                    

                    string url = "https://jsonplaceholder.typicode.com/posts/1";
                    var response = httpClient.GetAsync(url).Result;
                    string jsonResult = response.Content.ReadAsStringAsync().Result;   
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
            }
        }
    }
}

The call to GetAsync hangs throwing an exception with the following message:

System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

However, switch to .Net Core 2.0 and it works fine...

NOTE

I've tried using:

HttpClientFactory -> Same result
WebRequest        -> Same result

Thoughts?

UPDATE 1 This works when not on the corporate network which might mean a change in behavior with the proxy perhaps. However, core2.0 still works regardless so trying to find the difference.

UPDATE 2 Looks like a bug was introduced and it is reported...

https://github.com/dotnet/corefx/issues/30166#issuecomment-395489603

Couloir

One option is to use the new HttpClientFactory which is a upgrade/patch (remade and solved the issues with httpClient) on the HttpClient. check this out stevejgordon.co.uk/introduction-to-httpclientfactory-aspnetcore – Messily

I've tried with the HttpClientFactory too, same result. Also tried old WebRequest, same result – Couloir

I've tried async, doesn't help. I left it out for simplicity. – Couloir

It's entirely possible its our corp proxy but I don't know why .net Core 2.0 would work and .Net Core 2.1 would not – Couloir

Just tried the "jsonplaceholder.typicode.com/posts/1". Could not connect either.. retried, third time it worked. – Messily

Going to try from home, maybe it is our corp network.. – Couloir

@Marcus Höglund interesting that it did fail a few times though for you. – Couloir

It fails every time for me. – Couloir

ye, strange. I tried without the routes jsonplaceholder.typicode.com and after that it worked. Might be some handshake issue – Messily

I copy/paste your code into a console app and it works fine. i've started it 10x and didn't get any errors. @Couloir – Peyote

I tried this at home and it works. Seems like it is something with our corp network. However, that doesn't explain why core2.0 works and core2.1 does not. Going to look deeper into the changes in core2.1 – Couloir

Hey @Couloir I don't know if it might help, but Scott H. is mentioning something about occasionally HttpRequestException happening in 2.1 .. you might wanna have a lookt at it .. dunno if it really helps in your case.. hanselman.com/blog/… – Peyote

@Peyote I'll take a look! – Couloir

Is this bug fixed in net core 2.1.4 ? – Hawkbill

@Hawkbill I don't know, the workaround below works for us as we are a Windows only shop. But according this, it should be fixed. github.com/dotnet/corefx/issues/30191#issuecomment-401217223 – Couloir

Yes, workaround works for us too ;). – Hawkbill

Answer 1

So apparently there is a bug/breaking change reported on this.

Here: https://github.com/dotnet/corefx/issues/30166 https://github.com/dotnet/corefx/issues/30191

Two separate but related issues that I believe is what I am experiencing.

However, I found what appears to be a workaround.

using System;
using System.Diagnostics;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;

namespace TestHttpClient
{
    static class Program
    {
        static async Task Main(string[] args)
        {
            try
            {
                using (var httpClient = new HttpClient(new WinHttpHandler() { WindowsProxyUsePolicy = WindowsProxyUsePolicy.UseWinInetProxy }))
                {
                    httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

                    string url = "https://jsonplaceholder.typicode.com/posts/1";                   

                    var response = await httpClient.GetAsync(url);
                    string jsonResult = await response.Content.ReadAsStringAsync();
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                throw;
            }
        }
    }
}

The key part here is to use WinHttpHandler and set the WindowsProxyUsePolicy to WindowsProxyUsePolicy.UseWinInetProxy

WinHttpHandler is found by adding nuget package System.Net.Http.WinHttpHandler

Couloir

Unfortunately this only works for my problem when using System.Net.Http.WinHttpHandler 4.6.0-preview1. This package is provided by the daily build nuget store dotnet.myget.org/F/dotnet-core/api/v3/index.json . – Ellingson

@ToniWenzel Where can I get this version? The closest I found on NuGet was 4.6.0-preview.18571.3 but it doesn't work – Palmation

@LovyI currently use 4.6.0-preview1-26627-01. dotnet.myget.org/feed/dotnet-core/package/nuget/… – Ellingson

Well I simply forgot to add nuget daily build package source. Still doesn't fix my problem though – Palmation

Answer 2

The was a change in CoreFx 2.1 that causes the HttpClient to use a new HttpClientHandler. This is possibly the cause of your problem and why downgrading works.

There are many ways to reset the handler and you can read more about it in the change log. You can use the old HttpHandler by instantiating an HttpClient with a WinHttpHandler as the parameter, setting the environment variable DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER to false, or by calling the following in your code:

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

Ramulose

My code works fine at home. It appears to only be on the corp network. However, I tried your suggestions and it still doesn't work on the corp network. – Couloir

Is this still necessary in latest net core 2.1? I still have this temporarily in my code so everything works ok. – Hawkbill

I have tried to remove this line in our net core 2.2 app, but there are still same issues as with net core 2.1. Can anyone confirm? – Hawkbill

I'm still seeing this on dotnet core 3.1. Turning off UseSocketsHttpHandler increased my performance from minutes to mere seconds on macos – Merryman

Answer 3

This is a famous "21 seconds" timeout problem... What helps in Azure for service with rare calls (my service is calling external service from Azure infrastructure) is adding:

httpClient.DefaultRequestHeaders.ConnectionClose = true;

Glairy

More links regarding this "famous 21 seconds timeout problem" pls? I think I'm having it too. HttpRequestException and SocketException from Azure functions (behind apim) to downstream/source systems via HttpClient – Hausfrau

you can google for 21 seconds delay timeout, for ex getin2me.blogspot.com/2010/08/… – Glairy