Is it possible create a scalable WCF service with thousands of long-running TCP connections?

Question

I'm attempting to create a WCF service where several thousand (~10,000) clients can connect via a duplex NetTcpBinding for extended periods of time (weeks, maybe months).

After a bit of reading, it looks like it's better to host in IIS than a custom application or Windows service.

Is using WCF for such a service acceptable, or even possible? If so, where can I expect to run into throttling or performance issues, such as increasing the WCF ListenBacklog & MaxConcurrentConnections?

Thanks!

Sherry

I'n not a WCF expert, but I thought a Windows service would be more appropriate for hosting long-running connections? – Penmanship

I doubt if WCF can do that. It's a performance killer to my experience. – Quinn

@Mitch Wheat - WCF is just abstraction layer. WCF can easily be hosted by a Windows Service and use very low level binary serialization. – Strictly

So IIS would not be appropriate in this situation, rather, a Windows service? – Sherry

@Xaqron: I'm assuming you are referring to hosting in IIS? loads of WCF services are hosted fine in a windows service. @Alex Aza : I know that. I'm just not what I would term an 'expert'. – Penmanship

@Mitch: I had a experience with WCF hosted in IIS. I expected ~1000 concurrent connections but surprisingly it gets to it's knees at ~200 connections. – Quinn

Answer 1

Why do you need to maintain opened connection for weeks / months? That will introduce a lot of complexity, timeouts handling, error handling, recreating connection, etc. I even doubt that this will work.

Net.tcp connections use transport session which leads to PerSession instancing of WCF service - the single service instance servers all requests and lives for the whole duration of the session (weeks or months in your case) = instance and whole its content is still in the memory. Any interruption or unhandled exception will break the channel and close the session = all session's local data are lost and client must crate new proxy to start a new session again. Also any timeout (default is 20 minutes of inactivity) will close the session. For the last - depending of business logic complexity you can find that if even few hundreds clients needs processing in the same time single server is not able to serve all of them and some clients will timeout (again breaks the session). Allowing load balancing with net.tcp demands load balancing algorithm with sticky sessions (session affinity) and whole architecture becomes even more complicated and fragile. Scalability in net.tcp means that service can be deployed on multiple servers but the whole client session must be handled by single server (if server dies all sessions served by the server die as well).

Hosting in IIS/WAS/AppFabric has several advantages where two of them is health monitoring and process recycling. Health monitoring continuously check that worker process is still alive and can process request - if it doesn't it silently starts new worker process and routes new incoming requests to that process. Process recycling regularly recycles (default setting is after 29 hours) application domain which makes process healthy and reducing memory leaks. The side effect is that both recreating process or application domain will kill all sessions. Once you self host the service you lose all of them so you have to deal with health of your service yourselves.

Edit:

IMHO health status information doesn't have to be send over TCP. That is information that doesn't require all the fancy stuff. If you lose some information it will not affect anything = you can use UDP for health status transfers.

When using TCP you don't need to maintain proxy / session opened just to keep opened the connection. TCP connection is not closed immediately when you close the proxy. It remains opened in a pool for short duration of time and if any other proxy needs connection to the same server it is reused (the default idle timeout in pool should be 2 minutes) - I discussed Net.Tcp transport in WCF in another answer.

I'm not a fan of callbacks - this whole concept in WCF is overused and abused. Keeping 10.000 TCP connection opened for months just in case to be able to send sometimes data back to few PCs sounds ridiculous. If you need to communicate with PC expose the service on the PC and call it when you need to send some commands. Just add functionality which will call the server when the PC starts and when the PC is about to shut down + add transfering monitoring informations.

Anyway 10.000 PCs sending information every minute - this can cause that you will receive 10.000 requests in the same time - it can have the same effect as Denial of service attack. Depending on the processing time your server(s) may not be able to process them and many requests will timeout. You can also think about some message queuing or publish-subscribe protocols. Messages will be passed to a queue or topic and server(s) will process them continuously.

Argue

I would like to maintain the connections for as long as possible, as these are PC's that my company manages and we are looking to send back health statistics to our data center. We manage around 10,000 PC's and we need nearly real-time information on their health status. I'm afraid of the overhead each client PC would bring, consistently opening & closing connections to the server every 1 minute. If you have any better suggestions I would greatly appreciate it. Thanks! – Sherry

I added some ideas to the answer. – Argue

The long polling pattern requires connections to be open for quite awhile - that's the answer to your why, or at least this is one example that would require it. – Polly

Long polling pattern? What polling? – Argue

Well I guess you answered my question--no, not possible.. Thanks! – Sherry

I don't say it is not possible but I doubt that it will work as is without investing a lot of effort to all the stuff to make it work. You can always make some test. Create simple client and server with TCP connection and let it be opened for a week (and just work on another tasks). – Argue

If I'm understanding you correctly, this "Net.tcp connections use transport session which leads to PerSession instancing of WCF service" is definitely incorrect. You can use a singleton backing instance for a WCF service that lives beyond the individual session. And you can host it in IIS. – Tamatamable

@Joshua: That is not incorrect. That is how the service behaves by default when you don't change its instancing. And even if you change instancing to singleton you don't solve the transport session problems like timeouts. Also I didn't write that it is not possible to host net.tcp in IIS - I discussed benefits of IIS/WAS in the answer. – Argue

Right, wasn't sure if you were talking about the defaults or saying that it was the default behavior. – Tamatamable