Windows Azure Caching while debugging - Request Timeout

Question

I have a problem while using Windows Azure Caching with emulator (i.e., on Local, not Cloud).

My system configurations are:

• Windows 8
• Visual Studio 2012 Update 1
• Azure SDK and Emulator 1.8
• Windows Azure Caching 1.8.1
• Web API
• .NET 4.5

The problem is: Our server utilizes (in-role) Windows Azure Caching for session managements and etc. Once the server gets requests from clients, it starts communicating with Windows Azure Caching to put/get some objects (e.g., session objects). Then a "time-out" exception (Microsoft.ApplicationServer.Caching.DataCacheException) is occurred and the server becomes unresponsive, showing the following messages in the Output log of Visual Studio.

...
iisexpress.exe Error: 0 : ERROR: <DistributedCache.RoutingClient> 25cb6980-94c5-4e72-  adee-cd58defff7fe:SendMsgAndWait: Request TimedOut, msgId = 77

A first chance exception of type 'Microsoft.ApplicationServer.Caching.DataCacheException' occurred in Microsoft.ApplicationServer.Caching.Client.dll

'iisexpress.exe' (Managed (v4.0.30319)): Loaded 'C:\windows\assembly\GAC_MSIL\Microsoft.VisualStudio.Debugger.Runtime\11.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.Debugger.Runtime.dll'
iisexpress.exe Error: 0 : The thread '<No Name>' (0x16d0) has exited with code 0 (0x0).

The thread '<No Name>' (0x1a00) has exited with code 0 (0x0).
The thread '<No Name>' (0x2298) has exited with code 0 (0x0).

ERROR: <DistributedCache.RoutingClient> e91ff46b-870c-4d83-8eef-6bdb718baab9:SendMsgAndWait: Request TimedOut, msgId = 78

iisexpress.exe Warning: 0 : Microsoft.WindowsAzure.ServiceRuntime Verbose: 500 : Role instance status check starting

Microsoft.WindowsAzure.ServiceRuntime Verbose: 502 : Role instance status check succeeded: Ready

WARNING: <SimpleSendReceiveModule> DeadServerCallback Called, Server URI: [net.tcp://127.255.0.0:20004], Underlying exception - 

A first chance exception of type 'Microsoft.ApplicationServer.Caching.DataCacheException' occurred in Microsoft.ApplicationServer.Caching.Client.dll

iisexpress.exe Warning: 0 : WARNING: <SimpleSendReceiveModule> DeadServerCallback Called, Server URI: [net.tcp://127.255.0.0:20004], Underlying exception - 

iisexpress.exe Warning: 0 : WARNING: <SimpleSendReceiveModule> DeadServerCallback: Matches My Server, Cleaning Pending Requests

iisexpress.exe Warning: 0 : WARNING: <SimpleSendReceiveModule> 48c1dd7f-f54d-40a8-9990-25af48b91eb1:Request - 79, result - Status=SendFailed[System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host

   at System.Net.Sockets.Socket.Send(IList`1 buffers)

   at Microsoft.ApplicationServer.Caching.TcpSocketChannel.Send(IList`1 buffers, TimeSpan timeout)]

iisexpress.exe Warning: 0 : WARNING: <SimpleSendReceiveModule> DeadServerCallback: Matches My Server, Cleaning Pending Requests

A first chance exception of type 'Microsoft.ApplicationServer.Caching.DataCacheException' occurred in Microsoft.ApplicationServer.Caching.Client.dll

iisexpress.exe Warning: 0 : WARNING: <DistributedCache.SocketClientChannel.1> Request 81 to host net.tcp://127.255.0.0:20004/ failed Status=ChannelOpening

iisexpress.exe Warning: 0 : WARNING: <DistributedCache.SocketClientChannel.1> Request 80 to host net.tcp://127.255.0.0:20004/ failed Status=ChannelOpening

iisexpress.exe Warning: 0 : WARNING: <SimpleSendReceiveModule> 8ec8ff74-1f04-49da-8ff5-529d0697542f:Request - 81, result - Status=ChannelOpening

iisexpress.exe Warning: 0 : A first chance exception of type 'Microsoft.ApplicationServer.Caching.DataCacheException' occurred in Microsoft.ApplicationServer.Caching.Client.dll

When this starts occurring, the server is just down (i.e., running, but not responsive).

Any idea why this happens? Is it related to Windows 8?

Thanks,

Dynamite

Answer 1

I also had this problem and traced it down to the Hyper-V Virtual Ethernet Adapters on one machine. Disabling the two adapters I had installed fixed the timeout errors in Azure Caching. On this machine I could continue to stay connected to the Network/Internet.

On another machine Hyper-V wasn't installed, and disabling the Ethernet Adapter (disconnecting from the internet) "fixed" the issue. So to run my application I must first disconnect from the network each time.

On both of these machines the development Azure caching appears to have a problem when doing an IP look-up for the cache servers (e.g. 127.255.0.0, 127.255.0.1, 127.255.0.2, etc), it works for a few requests, then starts timing out. When disconnected from the network it always works.

This was working fine for me on both Windows 8 machines for months before I started having issues.

Update

Flushing the DNS cache fixed this for me, now it works while connected to a network. ipconfig /flushdns

Foghorn

Thanks for sharing your experience! :) – Dynamite

Answer 2

I haven't been able to fully confirm this, but the issue appears to be something to do with the legacy protocol in the azure caching dlls.

I was able to get around the issue by running this configuration on my default cache:

<dataCacheClient name="default" isCompressionEnabled="false" maxConnectionsToServer="4" useLegacyProtocol="false" >

Hope this helps

Wader