Only works on one-to-one of which was to be many-to-many, webrtc

Question

I am developing a conference style application (many-to-many) for video calls this style. The code is available on GitHub but I do not have much node.js experience, hence I decided to create my own server using PHP.

I created the server using WebSockets. It is simple - it receives messages and forwards them to all other connected clients (i.e., not the client that sent the message). Just that - nothing more; nothing less.

But my problem is that this architecture does not allow clients to connect with more than one person, i.e., when a client tries to connect with the third person, the additional streams fail. Clients can only connect one-to-one.

I do not know whether the error is in JavaScript or if I need to improve the server. What can I do to make it connect to all clients who join?

See my code:

HTML

<script type="text/javascript" src="http://127.0.0.1/scr/js/jquery.js"></script>

JavaScript

var Server = new WebSocket('ws://127.0.0.1:1805/'),
    myStream = null,
    peerConn = null,
    mediaConstraints = {
        'mandatory': {
            'OfferToReceiveAudio': true,
            'OfferToReceiveVideo': true
        }
    };


navigator.webkitGetUserMedia({
    audio: true,
    video: true
}, function(stream) {
    myStream = stream;

    $("body").append('<video width="320" height="240" muted="muted" autoplay="true" src="' + window.URL.createObjectURL(stream) + '"></video>');

    createPeerConnection();

    peerConn.addStream(myStream);
    peerConn.createOffer(function(sessionDescription) {
        peerConn.setLocalDescription(sessionDescription);
        console.log("Sending offer description");
        Server.send(JSON.stringify(sessionDescription));
    }, null, mediaConstraints);
}, function() {
    console.error('Error in my stream');
});

function createPeerConnection() {
    console.log('Creating peer connection');

    peerConn = new webkitRTCPeerConnection({
        'iceServers': [{
            'url': 'stun:stun.l.google.com:19302'
        }, {
            'url': 'turn:107.150.19.220:3478',
            'credential': 'turnserver',
            'username': 'subrosa'
        }]
    }, {
        'optional': [{
            'DtlsSrtpKeyAgreement': 'true'
        }]
    });

    peerConn.onicecandidate = function(event) {
        if (event.candidate) {
            Server.send(JSON.stringify({
                type: 'candidate',
                label: event.candidate.sdpMLineIndex,
                id: event.candidate.sdpMid,
                candidate: event.candidate.candidate
            }));
        } else {
            console.error('Candidate denied');
        }
    };
    peerConn.onaddstream = function(event) {
        console.log("Adding remote strem");
        $("body").append('<video width="320" height="240" autoplay="true" src="' + window.URL.createObjectURL(event.stream) + '"></video>');
    };
    peerConn.onremovestream = function(event) {
        console.log("Removing remote stream");
    };
}
Server.addEventListener("message", function(message) {
    var msg = JSON.parse(message.data);

    if(!myStream) {
        console.error('Error in my stream');
    }

    if (msg.type === 'offer') {
        createPeerConnection();

        console.log('Adding local stream...');

        peerConn.addStream(myStream);
        peerConn.setRemoteDescription(new RTCSessionDescription(msg));

        console.log("Sending answer to peer.");

        peerConn.createAnswer(function(sessionDescription) {
            peerConn.setLocalDescription(sessionDescription);
            Server.send(JSON.stringify(sessionDescription));
        }, null, mediaConstraints);
    } else if (msg.type === 'answer') {
        peerConn.setRemoteDescription(new RTCSessionDescription(msg));
    } else if (msg.type === 'candidate') {
        var candidate = new RTCIceCandidate({
            sdpMLineIndex: msg.label,
            candidate: msg.candidate
        });
        peerConn.addIceCandidate(candidate);
    }
}, false);

Delapaz

I can connect one-to-one but when I connect the third person, the third person connects only with the first. The first connects with the second and third. The second connecting with the first. The third connecting with the first. I'm lost in solving this impasse – Delapaz

Answer 1

The problem is that you're trying to use a single Peer Connection, but that will only work for a single connected party. You'll have to have an additional peer connection for each other party, and be able to associate websocket messages with users and a particular peer connection. You could do this yourself, or use a library like SimpleWebRTC that manages multiple user sessions for you.

Edit:

A very simplified explanation of how SimpleWebRTC works is this, which is one option for creating a mesh network of connected clients (all clients are connected to each other client):

1. A client joins a "room"
2. The client is notified about each client who has previously joined the room
3. For each other client, the client creates a new peer connection and stores it in an array of connected peers
4. When messages are received over the websocket, they must be associated with an Id, used to map to the proper peer connection

The critical difference in this architecture to yours is that you are creating a single peer connection, but you need to create, store, and track an array of peer connections, and you have to map your websocket messages to particular peers.

Shadchan

You could be more explanatory? I understand your main idea but want to know what to do in more specific. Would you like a way to fix my code... – Delapaz

@Delapaz "Would you like a way to fix my code..." No, your code can't be fixed because it is fundamentally flawed. It needs to be rewritten, or use a library that supports multiple connections. – Epner

you could e.g. make an array of peerconnections, index it with the remotepeerID for example, then in all exchanged messages, add a sourceID and PeerID to map the message to the right peerconnection in the array. – Telethermometer

@Epner I would not want to use a library for personal reasons, is you show me ways to reconstruct my code then? – Delapaz

@AlexGouaillard I can understand a little its logic, but that would be on the server or on the client code? It could be a little more descriptive? – Delapaz

@Delapaz Assuming that you wrote your original code, you seem more than capable of refactoring it to allow multiple users. On the other hand, if you copied and pasted it, I would not be doing you any favors by handing you more code to copy and paste. You would learn nothing, and end up with a mess of a project that you don't understand and are completely incapable of maintaining. Every developer who touches the project after you will curse your name, and my name for enabling you. – Epner

@Epner Sure! Quite understand what you're saying, but this code really is mine, so much so that I'm the updating as you can see in the edition of the question, but I am layman in WebRTC which means I do not know where to go. I advanced in the script and got something significant: I can connect one-to-one but when I connect the third person, the third person connects only with the first. The first connects with the second and third. The second connecting with the first. The third connecting with the first. I'm lost in solving this impasse .. – Delapaz

ok, great. If you want more help (or in the future), put the code public, and people can make pull requests. – Telethermometer

Answer 2

RTCPeerConnection is inherently a one-to-one connection between two clients (peers), so if you want to go beyond that you have to get clever.

The simplest step up is creating a mesh, essentially setting up one PeerConnection to each of the other participants, with all of the participants doing the same thing. You're going to hit a wall in client upload speed pretty fast this way though, usually topping out at 3-4 participants, typically based on the participant with the lowest upload speed.

For groups bigger than that you'd probably want some special setup like an MCU or Router solution, where essentially a special server acts as a super-participant that everyone connects to, which then either mixes together a video of everyone (usually with whomever is speaking as a larger video), or forwards everyone's video to everyone (since upload speed is usually the bottleneck).

Microseism

I understood what you said in theory but can not be applied in practice, you can help me? – Delapaz

I updated my code again in trying to achieve, I can connect one-to-one but when I connect the third person, the third person connects only with the first. The first connects with the second and third. The second connecting with the first. The third connecting with the first. I'm lost in solving this impasse – Delapaz

@Delapaz see my other answer on mesh. – Microseism