Forwarding RTSP stream from IP Camera to Browser in ASP.NET Core

Question

I've a Blazor hosted application where I need, upon client request, to get a h264 recording from an AXIS Camera (by means of RTSP PLAY command) and to return it in such a way that the browser can reproduce the video. If querying the AXIS camera for the list of recordings, the answer include this one, the one I'm trying to play on browser

    <recording diskid="SD_DISK" recordingid="20211109_122753_1AB3_B8A44F2D0300" starttime="2021-11-09T11:27:53.060281Z" starttimelocal="2021-11-09T12:27:53.060281+01:00" stoptime="2021-11-09T11:43:01.125987Z" stoptimelocal="2021-11-09T12:43:01.125987+01:00" recordingtype="continuous" eventid="continuous" eventtrigger="continuous" recordingstatus="completed" source="1" locked="No">
        <video mimetype="video/x-h264" width="800" height="600" framerate="15:1" resolution="800x600"/>
    </recording>

I can successfully reproduce the recording with VLC by means of "open network stream..." and typing in

rtsp://192.168.0.125/axis-media/media.amp?recordingid=20211109_140710_E1A3_B8A44F2D0300

and then providing username and password, so I'm sure the command is correct. By embedding username and password in the url, the recording can be played with this project as well, where a simpler syntax w.r.t the one I've used below is used, so my example is a bit overcomplicated probably.

Server side I can successfully retrieve the stream thanks to RtspClientSharp, but I'm not able to return it the correct way. So far I've this:

[HttpGet("RecordingsDemo")]
    public async Task<IActionResult> RecordingsDemo() {
        string deviceIp = "rtsp://192.168.0.125";
        string recordingUri = "rtsp://192.168.0.125/axis-media/media.amp?recordingid=20211109_140710_E1A3_B8A44F2D0300";
        Uri playRequestUri = new Uri(recordingUri);

        CancellationTokenSource cts = new CancellationTokenSource();
        NetworkCredential networkCredential = new NetworkCredential("user", "password");
        ConnectionParameters connectionParameters = new ConnectionParameters(new Uri(deviceIp), networkCredential);
        RtspTcpTransportClient RtspTcpClient = new RtspTcpTransportClient(connectionParameters);
        await RtspTcpClient.ConnectAsync(cts.Token);

        RtspRequestMessage message = new RtspRequestMessage(RtspMethod.SETUP, playRequestUri);
        message.AddHeader("Transport", "RTP/AVP/TCP;unicast");
        RtspResponseMessage response = await RtspTcpClient.EnsureExecuteRequest(message, cts.Token);
        System.Collections.Specialized.NameValueCollection headers = response.Headers;
        string sessionId = headers["SESSION"];
        if (sessionId == null) { throw new Exception("RTSP initialization failed: no session id returned from SETUP command"); }
        message = new RtspRequestMessage(RtspMethod.PLAY, playRequestUri, sessionId);
        response = await RtspTcpClient.EnsureExecuteRequest(message, cts.Token);
        Stream stream = RtspTcpClient.GetStream();
        if (stream != null) {
            Response.Headers.Add("Cache-Control", "no-cache");
            FileStreamResult result = new FileStreamResult(stream, "video/x-h264") {
                EnableRangeProcessing = true
            };
            return result;
        } else {
            return new StatusCodeResult((int)HttpStatusCode.ServiceUnavailable);
        }
        return new StatusCodeResult((int)HttpStatusCode.OK);
    }

Please note that in the code above I added a constructor to RtspRequestMessage in order to build it faster. In particular I added the following code:

    public uint _lastCSeqUsed { get; private set; }

    /// <param name="method">SETUP, PLAY etc</param>
    /// <param name="connectionUri">rtsp://<servername>/axis-media/media.amp?recordingid=...</param>
    /// <param name="cSeq">Method that generate the sequence number. The receiver will reply with the same sequence number</param>
    /// <param name="protocolVersion">Default to 1.0 if omitted or null</param>
    /// <param name="userAgent">Doesn't matter really</param>
    /// <param name="session">This parameter has to be initialized with the value returned by the SETUP method</param>
    public RtspRequestMessage(RtspMethod method, Uri connectionUri, string session = "", Func<uint> cSeqProvider = null, 
        Version protocolVersion = null, string userAgent = "client")
        : base((protocolVersion != null) ? protocolVersion : new Version("1.0"))
    {
        Method = method;
        ConnectionUri = connectionUri;
        UserAgent = userAgent;

        _cSeqProvider = (cSeqProvider != null) ? cSeqProvider : myfun;
        CSeq = (cSeqProvider != null) ? _cSeqProvider() : 0;

        if (!string.IsNullOrEmpty(session))
            Headers.Add("Session", session);
    }

    public void AddHeader(string name, string value)
    {
        Headers.Add(name, value);
    }

    private uint myfun()
    {
        return ++CSeq;
    }

When a client calls this method via GET method, I'm pretty sure the recording is correctly retrieved looking at the bandwitdh and at Wireshark. You can see Wireshark output in the next picture, where 192.168.0.125 is the camera and 192.168.0.120 the server.

However it seems the file returned by the server is not playable. I cannot play the returned file or the stream even with VLC. The client-server communication is shown in the next picture, where 192.168.0.16 is the client and 192.168.0.51 is the server.

I'd need to be able to return a stream that the html5 video element can play.

Could you please point me in the right direction? Thanks

EDIT.: As you can see I've found a way, posted below. However I hoped for a better solution, without the need of writing on disk and without the delay added by the generation of.ts files. Thus I leave the question open if someone is willing to contribute.

Titanic

Answer 1

Finally I was able to reach the goal, even if not in the way I wanted. These are the steps I had to accomplish, I've detailed them below.

Generate HTTP stream from RTSP stream

• Generate .m3u8 and .ts files from RTSP stream.
• Return .m3u8 file from controller.
• Play returned .m3u8 file. This requires a javascript library.

Generate .m3u8 and .ts files from RTSP stream

I've used the following FFmpeg command to extract the video from RTSP stream in such a way that I could return it by means of the command

ffmpeg.exe -i rtsp://username:[email protected]/axis-media/media.amp?recordingid=20211109_122753_1AB3_B8A44F2D0300 -fflags flush_packets -max_delay 5 -flags -global_header -hls_time 3 -hls_list_size 0 -vcodec copy -y .\example.m3u8

Where I had to use -hls_list_size 0 because in my case I have to convert a recording, and since the user needs to be able to seek back and forward in the recording, I had to set "delete no downloaded .ts segments", please see FFmpeg documentation. I could take advantage of this .m3u8 player demo to check if the problem was with the video I generated or with something else. This video How to stream IP Camera RTSP stream into browser as HLS via NodeJS, FFMPEG and ReactJS helped me too.

Return .m3u8 file from controller

Here I had two problems: the request was blocked due to cross-origin header missing. Furthermore, once the browser retrieved the .m3u8 file, it expected to be able to ask the controller for the .ts files. So I had to structure the code like this:

[ApiController]
[Route("[controller]")]
public class CameraSystemController : ControllerBase {
    [HttpGet("Example")]
    public async Task<IActionResult> Example() {
        Response.Headers.Add("Access-Control-Allow-Origin", "*");
        return File(System.IO.File.OpenRead("Output/Video/example.m3u8"), "application/octet-stream", enableRangeProcessing: true);
    }

    [HttpGet("{tsFileName}")]
    public async Task<IActionResult> Example_GetTS(string tsFileName) {
        Response.Headers.Add("Access-Control-Allow-Origin", "*");
        return File(System.IO.File.OpenRead("Output/Video/" + tsFileName), "application/octet-stream", enableRangeProcessing: true);
    }
}

Where I have to say thanks to this article on CORS and this post on implementing dynamic controller action.

Play .m3u8 file in browser

Finally, in order to play .m3u8 file in browser I had to use this HLS javascript project, that I discovered thanks to this post.

An example of a working html page I made is the following:

<!DOCTYPE html>
<html>
<head>
<meta charset=utf-8 />
<title>Example</title>
  <link href="https://unpkg.com/video.js/dist/video-js.css" rel="stylesheet">
  <script src="https://unpkg.com/video.js/dist/video.js"></script>
  <script src="https://unpkg.com/videojs-contrib-hls/dist/videojs-contrib-hls.js"></script>
   
</head>
<body>
  <video id="my_video_1" class="video-js vjs-fluid vjs-default-skin" controls preload="auto"
  data-setup='{}'>
            <source src="http://localhost:5000/CameraSystem/Example" type="application/x-mpegURL">
  </video>
<script>
var player = videojs('my_video_1');
player.play();
</script>
</body>
</html>

Titanic

in your question you say you are using RtspClientCSharp, but I can not find the class RtspTransportTcpClient in it. After some search, I found this library: SharpRTSP, which seems closer to the library you use in your code. Can you confirm? – Arctic

It is not RtspTransportTcpClient, it is RtspTcpTransportClient, and it's found here github.com/BogdanovKirill/RtspClientSharp/blob/master/… Instead, you could have a problem with RtspRequestMessage, since I used a constructor I added myself. I've updated the question adding the code for the new constructor. – Titanic