Ways to capture incoming WebRTC video streams (client side)
Asked Answered
C

2

6

I am currently looking to find a best way to store a incoming webrtc video streams. I am joining the videocall using webrtc (via chrome) and I would like to record every incoming video stream to from each participant to the browser. The solutions I am researching are:

  • Intercept network packets coming to the browsers e.g. using Whireshark and then decode. Following this article: https://webrtchacks.com/video_replay/

  • Modifying a browser to store recording as a file e.g. by modifying Chromium itself

Any screen-recorders or using solutions like xvfb & ffmpeg is not an options due the resources constrains. Is there any other way that could let me capture packets or encoded video as a file? The solution must be working on Linux.

Cormac answered 9/1, 2018 at 14:11 Comment(0)
S
6

If the media stream is what you want a method is to override the browser's PeerConnection. Here is an example:

In an extension manifest add the following content script:

content_scripts": [
    {
        "matches": ["http://*/*", "https://*/*"],
        "js": ["payload/inject.js"],
        "all_frames": true,
        "match_about_blank": true,
        "run_at": "document_start"
    }
]

inject.js:

var inject = '('+function() { 
    //overide the browser's default RTCPeerConnection. 
    var origPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection;
    //make sure it is supported
    if (origPeerConnection) {

        //our own RTCPeerConnection
        var newPeerConnection = function(config, constraints) {
            console.log('PeerConnection created with config', config);
            //proxy the orginal peer connection
            var pc = new origPeerConnection(config, constraints);
            //store the old addStream
            var oldAddStream = pc.addStream;

            //addStream is called when a local stream is added. 
            //arguments[0] is a local media stream
            pc.addStream = function() {
                console.log("our add stream called!")
                //our mediaStream object
                console.dir(arguments[0])
                return oldAddStream.apply(this, arguments);
            }
            
            //ontrack is called when a remote track is added.
            //the media stream(s) are located in event.streams
            pc.ontrack = function(event) {
                console.log("ontrack got a track")
                console.dir(event);
            }
    
            window.ourPC = pc;
            
            return pc; 
        };

    ['RTCPeerConnection', 'webkitRTCPeerConnection', 'mozRTCPeerConnection'].forEach(function(obj) {
        // Override objects if they exist in the window object
        if (window.hasOwnProperty(obj)) {
            window[obj] = newPeerConnection;
            // Copy the static methods
            Object.keys(origPeerConnection).forEach(function(x){
                window[obj][x] = origPeerConnection[x];
            })
            window[obj].prototype = origPeerConnection.prototype;
        }
    });
    }
    
}+')();';
var script = document.createElement('script');
script.textContent = inject;
(document.head||document.documentElement).appendChild(script);
script.parentNode.removeChild(script);

I tested this with a voice call in google hangouts and saw that two mediaStreams where added via pc.addStream and one track was added via pc.ontrack. addStream would seem to be local media streams and the event object in ontrack is a RTCTrackEvent which has a streams object. which I assume are what you are looking for.

To access these streams from your extenion's content script you will need to create audio elements and set the "srcObject" property to the media stream: e.g.

pc.ontrack = function(event) {

    //check if our element exists
    var elm = document.getElementById("remoteStream");
    if(elm == null) {
        //create an audio element
        elm = document.createElement("audio");
        elm.id = "remoteStream";

    }

    //set the srcObject to our stream. not sure if you need to clone it
    elm.srcObject = event.streams[0].clone();
    //write the elment to the body
    document.body.appendChild(elm);

    //fire a custom event so our content script knows the stream is available.
    // you could pass the id in the "detail" object. for example:
    //CustomEvent("remoteStreamAdded", {"detail":{"id":"audio_element_id"}})
    //then access if via e.detail.id in your event listener.
    var e = CustomEvent("remoteStreamAdded");
    window.dispatchEvent(e);

}

Then in your content script you can listen for that event/access the mediastream like so:

window.addEventListener("remoteStreamAdded", function(e) {
    elm = document.getElementById("remoteStream");
    var stream = elm.captureStream();
})

With the capture stream available to your content script you can do pretty much anything you want with it. For example, MediaRecorder works really well for recording the stream(s) or you could use something like peer.js or maybe binary.js to stream to another source.

I haven't tested this but it should also be possible to override the local streams. For example, in the inject.js you could establish some blank mediastream, override navigator.mediaDevices.getUserMedia and instead of returning the local mediastream return your own mediastream.

This method should work in firefox and maybe others as well assuming you use an extenion/app to load the inject.js script at the start of the document. It being loaded before any of the target's libs is key to making this work.

Speak answered 28/2, 2018 at 4:7 Comment(3)
Is this answer still relevant? I've tried your inject.js method in Google Meet and the newPeerConnection function doesn't get invokedWinsor
@nomadcrypto huge thanks for the answer. Can you please share a bit more code if it's in github or anywhere else?Triecious
@nomadcrypt Could you provide an update ? Most of the methods used are now deprecatedVesical
C
0

Capturing packets will only give you the network packets which you would then need to turn into frames and put into a container. A server such as Janus can record videos.

Running headless chrome and using the javascript MediaRecorder API is another option but much more heavy on resources.

Crucifix answered 9/1, 2018 at 15:44 Comment(2)
As far I understand what Janus does, that would require to have control over the webrtc server. I would like to record a call as a participant to external video call without any control over the server. Is there a way to somehow proxy e.g. my local traffic to Janus from my browser instance?Cormac
if you have a browser instance use the MediaRecorder API, see here for an example. You'll need to use Chromes tabcapture to capture the tab and add audio for the local and remote participants.Crucifix

© 2022 - 2024 — McMap. All rights reserved.