Play MPEG-2 TS using MseStreamSource

Question

I need to display a live video stream in a UWP application.

The video stream comes from a GoPro. It is transported by UDP messages. It is a MPEG-2 TS stream. I can play it successfully using FFPlay with the following command line :

ffplay -fflags nobuffer -f:v mpegts udp://:8554

I would like to play it with MediaPlayerElement without using a third party library.

According to the following page : https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/supported-codecs UWP should be able to play it. (I installed the "Microsoft DVD" application in the Windows Store).

I receive the MPEG-2 TS stream with a UdpClient. It works well. I receive in each UdpReceiveResult a 12 bytes header, followed by 4, 5, 6, or 7 MPEGTS packets (each packet is 188 bytes, beginning with 0x47).

I created a MseStreamSource :

_mseStreamSource = new MseStreamSource();
_mseStreamSource.Opened += (_, __) =>
{
    _mseSourceBuffer = _mseStreamSource.AddSourceBuffer("video/mp2t");
    _mseSourceBuffer.Mode = MseAppendMode.Sequence;
};
_mediaPlayerElement.MediaSource = MediaSource.CreateFromMseStreamSource(_mseStreamSource);

This is how I send the messages to the MseStreamSource :

    UdpReceiveResult receiveResult = await _udpClient.ReceiveAsync();
    byte[] bytes = receiveResult.Buffer;
    mseSourceBuffer.AppendBuffer(bytes.AsBuffer());

The MediaPlayerElement displays the message "video not supported or incorrect file name". (not sure of the message, my Windows is in French).

Is it a good idea to use the MseAppendMode.Sequence mode ? What should I pass to the AppendBuffer method ? The raw udp message including the 12 bytes header or each MPEGTS 188 bytes packet ?

Parrisch

As the document mentioned, "Where MPEG-2/MPEG-1 support is indicated, it is only supported with the installation of the optional Microsoft DVD Universal Windows app." could you please try to install the app? And by the way, did all the packets throw the exception or only parts? – Goldfarb

If I look for "Microsoft DVD" on the Windows Store, it gives me an application called "Extension vidéo MPEG-2". My Windows is in french. I tried to install this app. But it doesn't changes anything. About the exceptions, it is only for some packets. – Melee

I finally found the Microsoft DVD application. I installed it, but it doesn't changes anything. – Melee

I edited my question. I found that UDP buffers contains correct MPEGTS packets. Also discovered the MseAppendMode.Sequence mode. But it doesn't changes anything. – Melee

Answer 1

I finally got the video working !

Here are the steps I follow to extract the MPEG-TS packets and correctly send them to the MseStreamSource :

The MseSourceBuffer needs to be in "Sequence" mode :

_mseSourceBuffer.Mode = MseAppendMode.Sequence;

For each received UDP datagram, I extract the MPEG-TS packets. To do that, I ignore the first 12 bytes of the UDP datagram. Then I extract each 188 bytes packet in a separate array (each packet starts with 0x47).

I send each packet to a synchronized queue.

I dequeue the packets from the queue and send them grouped to the MseSourceBuffer. I create a new group for each PAT packet (pid = 0) :

byte[] bytes;
// [...] combine the packets of the group
mseSourceBuffer.AppendBuffer(bytes.AsBuffer());

I tried to use a MemoryStream and call the AppendStream() method, but with no success.

Also care about threads synchronization : packets order should not be lost. That is the reason for the synchronized queue.

Hope it can help someone else.

This wikipedia MPEG-TS page helped me a lot.

Parrisch