Live FLV streaming in C# WebApi

Question

Currently I have a working live stream using webapi. By receiving a flv stream directly from ffmpeg and sending it straight to the client using PushStreamContent. This works perfectly fine if the webpage is already open when the stream starts. The issue is when I open another page or refresh this page you can no longer view the stream (the stream is still being sent to the client fine). I think it is due to something missing from the start of the stream but I am not sure what to do. Any pointers would be greatly appreciated.

Code for client reading stream

public class VideosController : ApiController
{
    public HttpResponseMessage Get()
    {
        var response = Request.CreateResponse();
        response.Content = new PushStreamContent(WriteToStream, new MediaTypeHeaderValue("video/x-flv"));

        return response;
    }

    private async Task WriteToStream( Stream arg1, HttpContent arg2, TransportContext arg3 )
    {
        //I think metadata needs to be written here but not sure how
        Startup.AddSubscriber( arg1 );
        await Task.Yield();
    }
}

Code for receiving stream and then sending to client

while (true)
{
    bytes = new byte[8024000];
    int bytesRec = handler.Receive(bytes);

    foreach (var subscriber in Startup.Subscribers.ToList())
    {
        var theSubscriber = subscriber;
        try
        {
            await theSubscriber.WriteAsync( bytes, 0, bytesRec );
        }
        catch
        {
            Startup.Subscribers.Remove(theSubscriber);
        }
    }
}

Marabout

Answer 1

I've never used FLV or studied video formats closely

Most file formats are structured, especially video formats. They contain frames (i.e. a complete or partial screen shots depending on the compression format).

You should be really lucky if you manage to hit a specific frame when you start streaming to the new subscriber. Hence when they start receiving the stream they cannot identify the format as frame is partial.

You can read more FLV frames in wikipedia article. This is most likely your problem.

A simple attempt would be to try to save the initial header that you receive from the streaming server when the first subscriber connects.

Something like:

static byte _header = new byte[9]; //signature, version, flags, headerSize

public void YourStreamMethod()
{
    int bytesRec = handler.Receive(bytes);
    if (!_headerIsStored)
    {
        //store header
        Buffer.BlockCopy(bytes, 0, _header, 0, 9);
        _headerIsStored = true;
    }
}

.. which allows you to send the header to the next connecting subscriber:

private async Task WriteToStream( Stream arg1, HttpContent arg2, TransportContext arg3 )
{
    // send the FLV header
    arg1.Write(_header, 0, 9);

    Startup.AddSubscriber( arg1 );
    await Task.Yield();
}

Once done, pray that the receiver will ignore partial frames. If it doesn't you need to analyze the stream to identify where the next frame is.

To do that you need to do something like this:

1. Create a BytesLeftToNextFrame variable.
2. Store the received packet header in a buffer
3. Convert the "Payload size" bits to an int
4. Reset the BytesLeftToNextFrame to the parsed value
5. Countdown until the next time you should read a header.

Finally, when a new client connects, do not start streaming until you know that the next frame arrives.

Pseudo code:

var bytesLeftToNextFrame = 0;
while (true)
{
    bytes = new byte[8024000];
    int bytesRec = handler.Receive(bytes);

    foreach (var subscriber in Startup.Subscribers.ToList())
    {
        var theSubscriber = subscriber;
        try
        {
            if (subscriber.IsNew && bytesLeftToNextFrame < bytesRec)
            {
                //start from the index where the new frame starts
                await theSubscriber.WriteAsync( bytes, bytesLeftToNextFrame, bytesRec - bytesLeftToNextFrame);
                subscriber.IsNew = false;
            }
            else
            {
                //send everything, since we've already in streaming mode
                await theSubscriber.WriteAsync( bytes, 0, bytesRec );
            }
        }
        catch
        {
            Startup.Subscribers.Remove(theSubscriber);
        }
    }

    //TODO: check if the current frame is done
    // then parse the next header and reset the counter.
}

Hsu

Found a simple FLV metadata reader: johndyer.name/flash-flv-meta-reader-in-net-c – Hsu

Answer 2

I'm not a expert in streaming, but looks like you should close stream then all data will be writed

await theSubscriber.WriteAsync( bytes, 0, bytesRec );

Like it mentions in WebAPI StreamContent vs PushStreamContent

{
     // After save we close the stream to signal that we are done writing.
     xDoc.Save(stream);
     stream.Close();
}

Roberge

The problem is this is a live stream and I want keep it live when new people connect – Marabout

Answer 3

I LIKE THIS CODE BECAUSE IT DEMONSTRATES A FUNDAMENTAL ERROR when dealing with async programming

while (true)
{

}

this is a synced loop, that loops itself as fast as possible.. every second it can execute thousands of times (depending on availabe software and hardware resources)

await theSubscriber.WriteAsync( bytes, 0, bytesRec );

this is an async command (if that wasn't clear enough) that execute in a DIFFERENT thread (while loop representes the main thread execution)

now... in order to make the while loop to wait to the async command we use await... sounds good (or else the while loop will execute thousands of times, executing countless async commands)

BUT because the loop (of subscribers) need to transmit the stream for all subscribers simulatanly it get stucked by the await keyword

THAT IS WHY RELOAD / NEW SUBSCRIBER FREEZE THE WHOLE THING (new connection = new subscriber)

conclusion: the entire for loop should be inside a Task. the Task need to wait until the server send the stream to all subscribers. ONLY THEN it should continue to the while loop with ContinueWith (that is why it called like that, right?)

so... the write command need to get execute without await keyword

theSubscriber.WriteAsync

the foreach loop should use a task that continue with the while loop after it is done

Eridanus

a synced loop? What are you talking about? It's a loop. period. Its what you do in it that's important. He do use await in the loop, thus it's perfectly valid. – Hsu