Cause of high UDP package loss on localhost?

Question

In my WPF 4.0 application, I have a UDP listener implemented as shown below. On my Windows 7 PC, I'm running both server and client on localhost.

Each received datagram is a scanline of a larger bitmap, so after all scanlines have been received the bitmap is shown on the UI thread. This seems to work. However, occasionally some 1-50% scanlines are missing. I would expect this on a weak network connection, but not when run locally.

What may cause UDP package loss with the following piece of code?

IPEndPoint endPoint = new IPEndPoint(IPAddress.Any, PORT);
udpClient = new UdpClient(endPoint);
udpClient.Client.ReceiveBufferSize = 65535; // I've tried many different sizes...

var status = new UdpStatus()
{
    u = udpClient,
    e = endPoint
};

udpClient.BeginReceive(new AsyncCallback(UdpCallback), status);

private void UdpCallback(IAsyncResult ar)
{
    IPEndPoint endPoint = ((UdpStatus)(ar.AsyncState)).e;
    UdpClient client = ((UdpStatus)(ar.AsyncState)).u;

    byte[] datagram = client.EndReceive(ar, ref endPoint);

    // Immediately begin listening for next packet so as to not miss any.
    client.BeginReceive(new AsyncCallback(UdpCallback), ar.AsyncState);

    lock (bufferLock)
    {
        // Fast processing of datagram.
        // This merely involves copying the datagram (scanline) into a larger buffer.
        //
        // WHEN READY:
        // Here I can see that scanlines are missing in my larger buffer.
    }
}

If I put a System.Diagnostics.Debug.WriteLine in my callback, the package loss increases dramatically. It seems that a small millisecond delay inside this callback causes problems. Still, the same problem is seen in my release build.

UPDATE

The error becomes more frequent when I stress the UI a bit. Is the UdpClient instance executed on the main thread?

Picrite

How much data, or how many packets per second are you sending ? It's quite easy to overflow the receiving queue if you're not reading data fast enough. – Landed

are you sure you aren't seeing out-of-order lines? You could receive a new packet BEFORE the old one has been lock (bufferLock) { save }. I would but the BeginReceive inside the lock – Acidfast

Approximately what rate are you sending data? It might be an idea to post your sending code too. What happens if you switch to TCP instead of UDP? – Fere

UDP does not guarantee that your packets will arrive in order. Are you sure that this does not compromises your image processing? – Borghese

This is a video stream, so there is a LOT of data. Putting the BeginReceive inside the lock makes no difference. I cannot switch to TCP since I cannot change the server code. Packages in wrong order is a possibility, and it will happen - but, the problem here is that I see this error way too often. – Picrite

Would it help putting the UdpClient in a worker thread? – Picrite

Increasing the ReceiveBufferSize definitely yields a better result. Just how large should it be? – Picrite

Answer 1

To avoid the thread block issue, try this approach that uses the newer IO Completion port receive method:

private void OnReceive(object sender, SocketAsyncEventArgs e)
{
TOP:
    if (e != null)
    {
        int length = e.BytesTransferred;
        if (length > 0)
        {
            FireBytesReceivedFrom(Datagram, length, (IPEndPoint)e.RemoteEndPoint);
        }
        e.Dispose(); // could possibly reuse the args?
    }
    Socket s = Socket;
    if (s != null && RemoteEndPoint != null)
    {
        e = new SocketAsyncEventArgs();
        try
        {
            e.RemoteEndPoint = RemoteEndPoint;
            e.SetBuffer(Datagram, 0, Datagram.Length); // don't allocate a new buffer every time
            e.Completed += OnReceive;
            // this uses the fast IO completion port stuff made available in .NET 3.5; it's supposedly better than the socket selector or the old Begin/End methods
            if (!s.ReceiveFromAsync(e)) // returns synchronously if data is already there
                goto TOP; // using GOTO to avoid overflowing the stack
        }
        catch (ObjectDisposedException)
        {
            // this is expected after a disconnect
            e.Dispose();
            Logger.Info("UDP Client Receive was disconnected.");
        }
        catch (Exception ex)
        {
            Logger.Error("Unexpected UDP Client Receive disconnect.", ex);
        }
    }
}

Metritis

Interesting. I'll try this as soon as I can. Thanks! – Picrite

Could you please show the rest of the code? What does the initial call look like, where you set the up the OnReceive callback for the first time? – Picrite

To set it up initially, just call OnReceive(null, null). You can see it is getting Socket and RemoteEndPoint and DataGram from class variables. Those would need to be set up first. – Metritis

I'll accept this as an answer even though I haven't tested it yet. It's an alternative solution that adds value to the question. – Picrite

@Picrite Did this code help you? Can you post an example of using this function in your project(if it's possible)? Because there is some equivocation that impede properly understand code in the answer(maybe just for me). Thanks! – Brycebryn

I have since proven that you don't need to make a new args object for every receive call. Just reuse the same one for every call. – Metritis

Sorry, I haven't had time to test this thoroughly. Too much to do :) Though, I do find it interesting that at least a few more people have encountered this problem. – Picrite