Server design using SocketAsyncEventArgs

Question

I want to create an asynchronous socket Server using the SocketAsyncEventArgs event.

The server should manage about 1000 connections at the same time. What is the best way to handle the logic for each packet?

The Server design is based on this MSDN example, so every socket will have his own SocketAsyncEventArgs for receiving data.

1. Do the logic stuff inside the receive function. No overhead will be created, but since the next ReceiveAsync() call won’t be done before the logic has completed, new data can’t be read from the socket. The two main questions for me are: If the client sends a lot of data and the logic processing is heavy, how will the system handle it (packets lost because buffer is to full)? Also, if all clients send data at the same time, will there be 1000 threads, or is there an internal limit and a new thread can’t start before another one completes execution?

2. Use a queue. The receive function will be very short and execute fast, but you’ll have decent overhead because of the queue. Problems are, if your worker threads are not fast enough under heavy server load, your queue can get full, so maybe you have to force packet drops. You also get the Producer/Consumer problem, which can probably slow down the entire queue with to many locks.

So what will be the better design, logic in receive function, logic in worker threads or anything completely different I’ve missed so far.

Another Quest regarding data sending.

Is it better to have a SocketAsyncEventArgs tied to a socket (analog to the receive event) and use a buffer system to make one send call for a few small packets (let’s say the packets would otherwise sometimes! send directly one after another) or use a different SocketAsyncEventArgs for every packet and store them in a pool to reuse them?

Flowerage

Answer 1

To effectivly implement async sockets each socket will need more than 1 SocketAsyncEventArgs. There is also an issue with the byte[] buffer in each SocketAsyncEventArgs. In short, the byte buffers will be pinned whenever a managed - native transition occurs (sending / receiving). If you allocate the SocketAsyncEventArgs and byte buffers as needed you can run into OutOfMemoryExceptions with many clients due to fragmentation and the inability of the GC to compact pinned memory.

The best way to handle this is to create a SocketBufferPool class that will allocate a large number of bytes and SocketAsyncEventArgs when the application is first started, this way the pinned memory will be contiguous. Then simply resuse the buffers from the pool as needed.

In practice I've found it best to create a wrapper class around the SocketAsyncEventArgs and a SocketBufferPool class to manage the distribution of resources.

As an example, here is the code for a BeginReceive method:

private void BeginReceive(Socket socket)
    {
        Contract.Requires(socket != null, "socket");

        SocketEventArgs e = SocketBufferPool.Instance.Alloc();
        e.Socket = socket;
        e.Completed += new EventHandler<SocketEventArgs>(this.HandleIOCompleted);

        if (!socket.ReceiveAsync(e.AsyncEventArgs)) {
            this.HandleIOCompleted(null, e);
        }
    }

And here is the HandleIOCompleted method:

private void HandleIOCompleted(object sender, SocketEventArgs e)
    {
        e.Completed -= this.HandleIOCompleted;
        bool closed = false;

        lock (this.sequenceLock) {
            e.SequenceNumber = this.sequenceNumber++;
        }

        switch (e.LastOperation) {
            case SocketAsyncOperation.Send:
            case SocketAsyncOperation.SendPackets:
            case SocketAsyncOperation.SendTo:
                if (e.SocketError == SocketError.Success) {
                    this.OnDataSent(e);
                }
                break;
            case SocketAsyncOperation.Receive:
            case SocketAsyncOperation.ReceiveFrom:
            case SocketAsyncOperation.ReceiveMessageFrom:
                if ((e.BytesTransferred > 0) && (e.SocketError == SocketError.Success)) {
                    this.BeginReceive(e.Socket);
                    if (this.ReceiveTimeout > 0) {
                        this.SetReceiveTimeout(e.Socket);
                    }
                } else {
                    closed = true;
                }

                if (e.SocketError == SocketError.Success) {
                    this.OnDataReceived(e);
                }
                break;
            case SocketAsyncOperation.Disconnect:
                closed = true;
                break;
            case SocketAsyncOperation.Accept:
            case SocketAsyncOperation.Connect:
            case SocketAsyncOperation.None:
                break;
        }

        if (closed) {
            this.HandleSocketClosed(e.Socket);
        }

        SocketBufferPool.Instance.Free(e);
    }

The above code is contained in a TcpSocket class that will raise DataReceived & DataSent events. One thing to notice is the case SocketAsyncOperation.ReceiveMessageFrom: block; if the socket hasn't had an error it immediately starts another BeginReceive() which will allocate another SocketEventArgs from the pool.

Another important note is the SocketEventArgs SequenceNumber property set in the HandleIOComplete method. Although async requests will complete in the order queued, you are still subject to other thread race conditions. Since the code calls BeginReceive before raising the DataReceived event there is a possibility that the thread servicing the orginal IOCP will block after calling BeginReceive but before rasing the event while the second async receive completes on a new thread which raises the DataReceived event first. Although this is a fairly rare edge case it can occur and the SequenceNumber property gives the consuming app the ability to ensure that data is processed in the correct order.

One other area to be aware of is async sends. Oftentimes, async send requests will complete synchronously (SendAsync will return false if the call completed synchronously) and can severely degrade performance. The additional overhead of of the async call coming back on an IOCP can in practice cause worse performance than simply using the synchronous call. The async call requires two kernel calls and a heap allocation while the synchronous call happens on the stack.

Hope this helps, Bill

Send

Answer 2

In your code, you do this:

if (!socket.ReceiveAsync(e.AsyncEventArgs)) {
  this.HandleIOCompleted(null, e);
}

But it is an error to do that. There is a reason why the callback is not invoked when it finishes synchronously, such action can fill up the stack.

Imagine that each ReceiveAsync is always returning synchronously. If your HandleIOCompleted was in a while, you could process the result that returned synchronously at the same stack level. If it didn't return synchronously, you break the while. But, by doing the you you do, you end-up creating a new item in the stack... so if you have bad luck enough, you will cause stack overflow exceptions.

Biopsy

I am also seeing stack overflow exceptions with this pattern. Basically, if your handler for SendAsync() invokes a ReceiveAsync() and your handler for ReceiveAsync() invokes a SendAsync(), you will pop the stack if your client/server communication is very chatty. – Jacaranda