How to get multiple streams from WebRTC PeerConnection

Question

According to RTCPeerConnection.ontrack documentation, "ontrack" event suppose to fire for each incoming streams. I have a PeerConnection with two video streams, after connection, "ontrack" fires two times (up to here everything is OK). But both times it sends same stream out, so I end up with two identical video, I am sure sender is sending two different streams, dimension and frame rate of them are different and I can clearly see in chrome://webrtc-internals/ that 2 video streams have different frame size/rate.

Here is PeerConnection ontrack code:

        this.peerConnection.ontrack = function(evt) {
            console.log("PeerConnection OnTrack event: ", evt.streams);
            that.emit('onRemoteStreamAdded', evt.streams);
        };

I don't assume evt.streams has 1 object, so I did not write evt.streams[0].

Here is Chrome console log:

As it is obvious from log getRemoteStreams() returns only one object. How is it possible ontrack fires two time when it has only one stream, and why second RTCRtpTransceiver does not make a new stream?

Icj

Answer 1

I solved it after few hours of struggling with different browsers and reading documentations several times!

Problem starts at MediaStream.id, it suppose to be unique but <video> element in HTML5 only listens to first track inside each stream. PeerConnection adds new transceivers (as MediaStreamTrack) to same MediaStream, so no matter how many times ontrack handler fires, you get exact same MediaStream objects, but each time you have new unique MediaStreamTrack inside the RTCTrackEvent.

Solution is to create new MediaStream object for each new MediaStreamTrack inside ontrack handler.

this.peerConnection.ontrack = function(event) {
    that.emit('onRemoteStreamAdded', new MediaStream([event.track]));
};

Or, more like standard examples:

pc.ontrack = function(event) {
  document.getElementById("received_video").srcObject = new MediaStream([event.track]);
};

Icj

Question is old now, but just to mention: Philipp's not too wrong about only one audio and one video, but no more of each one in each stream. Therefore answer is correct however, in the sender part, if you add tracks to different streams using second parameter of pc.addTrack() it will be automatically created for you on receiving part. Look – Sewage

Answer 2

You get two tracks which are part of a single stream. You can see that in the event.track property, one of them should be audio, the other video.

See https://blog.mozilla.org/webrtc/the-evolution-of-webrtc/ for background information on how streams and tracks work.

Tropo

Thank you for not reading my question! First, I don't have two streams, I have more than 5. Second, all of them are videos without audio track, see my question. Third, this was not WebRTC fault, it is partially HTML <video> element implementation and also MediaStream implementation fault in browser. Check my answer for correct solution. – Icj

are you calling addTrack(track) on the other end of the connection instead of addTrack(track, stream)? That would explain the behaviour as in that case everything will go into a catchall MediaStream on the receiver. That is generally considered to be a not so good idea. – Tropo

Answer 3

This thread is too old but I will add this answer for the future visitors. In WebRTC application, on both the ends changes has to be made. On the sender side multiple videos has to be added to the track separately(not with the usual procedure). With normal procedure both the streams will be combined on the receiver end. On the receiver end, you need to check the event.stream and even.track and add them separately to the appropriate tags.

Joris