Updating .NET framework resulting in SQL timeouts

Question

We have an app which targets .NET 4.5.1, and this has remained unchanged.

However when we upgraded the .NET framework on the server from 4.5.1 -> 4.7.1, we started experiencing SQL timeouts several hours afterwards (the app target remained at 4.5.1).

"Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached."

Other servers which had the same treatment produced the issue also, so we looked for a breaking change in .NET, and found this article: https://blogs.msdn.microsoft.com/dataaccesstechnologies/2016/05/07/connection-timeout-issue-with-net-framework-4-6-1-transparentnetworkipresolution/

That article quotes a different exception type, but might be somewhat related. However I'd be stunned if our DNS lookup took longer than 500ms. Also I'd expect to see far more cases of this connection string config reported and used.

Our app is high traffic, but we're confident we're not leaking connections as this has never been an issues for years until we updated the .NET framework.

We're going to try applying this fix (and wait >24 hours to see the results), but is there anything else we could have missed? We're not confident this is the solution.

EDIT: Even after rolling .NET back to 4.5.1 and restart all servers, we're still seeing the problem. Nothing else has changed in the codebase, but we've yet to roll back a registry change which enabled 'SchUseStrongCrypto' - if that could be the cause?

Selfabuse

is your sql server up to date? there have been crypto changes that may lead to issues (though more likely with 4.7.2 instead of 4.7.1) – Bisson

We're using SQL Server 2012 (SP1) - 11.0.3, so I wouldn't have thought that would cause any connection pool issues? – Selfabuse

Have you recently moved to TLS 1.2? Can you try looking into TLS compatibility level? – Halfwitted

This would be better on serverfault.com – Vigil

There have been implementation changes in ADO.NET over the last years. Our service has been broken as well (through an automatic minor version nightly Windows Update...) but with other symptoms. This does not directly help you but you might be onto something. – Cirrate

who's your certificate issuer? and which version of windows is on for the webserver? – Disciple

Monitor whether the number of connections on your SQL Server keeps growing; the simplest way by regularly running SP_WHO and SP_WHO2. If so, the server suffers from threadpool starvation as the error indicates. The output of SP_WHO(2) most of the time includes an indication which program is involved. It might also be a complete other one. – Wiesbaden

Do yo have a time server on you network to synchronize the time? When networks are busy I see DNS lookups take longer periods of time but I don't think this would cause the issue to continue. If you rolled back to old version of Net make sure you delete the project bin folder to force all code to recompile. The dependencies in the compiler will not automatically recompile all the code when Net version changes. – Skywriting

If it is indeed a connection pool issue, then maybe something like this answer will help you identify it. – Vivle

Is there any news? – Joletta

Answer 1

I've not experienced this, but the link https://learn.microsoft.com/en-us/dotnet/framework/migration-guide/runtime/4.0-4.7.1 indicates a change to the SQL Connection Pool, where it now retries broken connections for much longer. The link also provides a setting to bypass the new behaviour;

 ConnectRetryCount = 0

Is is possible that connections in the pool are now staying alive much longer than previously as a side-effect or intended feature of this behaviour change, and therefore clogging up your connection pool with 'dead but retrying connections' whereas previously they would have died ?

Its a bit speculative; but might lead you down the right path.

Kym