What's the best way to STREAM LIVE WEBCAM to SERVER and BACK TO THE WEB?

Question

I need some help.

What is the best way to set up LIVE STREAMING over the web from my WEBCAM to the server and back to multiple users?

Essentially I'm trying to create a group video chat application that can support many users.

I don't want it to be peer to peer webRTC.

I actually managed to make it work with getUserMedia() -> mediaRecorder -> ondataavailable -> pass blob chunks to node.js via SOCKET.IO -> socket.io sends back blob chunks to other connected users -> append those chunks to a sourceBuffer that's connected to a mediaSource that's set as the source URL on a

And it actually worked! BUT it's so slow and laggy and resource intensive. As these chunks get passed like 20 per second and it's slowing the page a lot. I don't think you're supposed to pass that many blobs to the sourceBuffer so quickly. Just for a test I tried saving mediaRecordings every 3 seconds (so it's not that resource intensive) and passing those webm blobs to the sourceBuffer but for some reason only the first webm loads, and the other ones don't get added or start playing.

It just can't work for a production app this way.

What's the "RIGHT" way to do this?

How to pass a video stream from webcam to a Node.js server properly?

And how to stream this live stream back to the web from the Node.js server so that we can have a group video chat?

I'm a bit lost. Please help.

Do I use HLS? RecordRTC?

Do I stream from Node.js via http or via socket.io?

There are services that already let you do that easily like vonage video api tokbox but those seem to be very expensive?

I want to run the video streaming through my own Node.js server that I control.

What's the best way to do this?

Please help.

Thank you

Gober

Answer 1

Essentially I'm trying to create a group video chat application that can support many users.

I don't want it to be peer to peer webRTC.

Video chat requires low latency, and therefore requires usage of WebRTC. Remember that one of the "peers" can actually be a server.

And it actually worked! BUT it's so slow and laggy and resource intensive.

Video encoding/decoding is resource intensive no matter how you do it. If by "slow" and "laggy" you mean high latency, then yes, recording chunks, sending chunks, decoding chunks, will have higher latency by its very nature. Additionally, what you're describing won't drop frames or dynamically adjust the encoding, so if a connection can't keep up, it's just going to buffer until it can. This is a different sort of tradeoff than what you want.

Again, for a video chat, realtime-ness is more important than quality and reliability. If that means discarding frames, resampling audio stupid-fast to catch up, encoding at low bitrates, even temporarily dropping streams entirely for a few seconds, that's what needs to happen. This is what the entire WebRTC stack does.

As these chunks get passed like 20 per second and it's slowing the page a lot. I don't think you're supposed to pass that many blobs to the sourceBuffer so quickly.

No, this is unlikely your problem. The receiving end probably just can't keep up with decoding all these streams.

Do I use HLS?

Not for anyone actively participating in the chat... people who require low latency. For everyone else, yes you can utilize HLS and DASH to give you a more affordable way to distribute your stream over existing CDNs. See this answer: https://mcmap.net/q/489476/-low-latency-lt-2s-live-video-streaming-html5-solutions Basically, scrutinize your requirements and determine if everyone is actually participating. If they aren't, move them to a cheaper streaming method than WebRTC.

RecordRTC?

No, this is irrelevant to your project and frankly I don't know why people keep using this library for anything. Maybe they have some specific use case for it I don't know about, but browsers have had built-in MediaRecorder for years.

There are services that already let you do that easily like vonage video api tokbox but those seem to be very expensive?

This is an expensive thing to do. I think you'll find that using an existing service that already has the infrastructure ready to go is going to be cheaper than doing it yourself in most cases.

Xenophanes

Thanks from my further research I came to the conclusions you pointed me to. I need to use webRTC and have a webRTC server to handle video transfer for a group of people. I only need to have 5-10 people connected at a time but if it's peer-to-peer it won't work that's why I was discounting webRTC but I missed the part where you can have a webRTC server. Do you have any recommendations for that? I managed to improve my working code now it sends chunks in 1 sec intervals and displays them in a new video tag on top of the old one lol. That works by yeah I have 1 second latency.. – Gober

Answer 2

For anyone needing another approach, You may use Ffmpeg and Socket to do this.

First, install Ffmpeg Then implement following:

There are 2 parts:

• On the server capture your PC webcam feed using Ffmpeg and transmit the frames over socket.

const express = require('express');
const app = express();
const http = require('http').Server(app);
const io = require('socket.io')(http/*, {origins: allowedOrigins} */);
const port = 5000;

var ffmpeg = require('child_process').spawn("ffmpeg", [
    "-i", "/dev/video0", 
    "-vb", "5M",
    "-preset", "ultrafast", 
    '-acodec', 'copy',
    "-f", "mjpeg", 
    "pipe:1"]);

ffmpeg.on('error', function (err) {
    throw err;
});

ffmpeg.on('close', function (code) {
    console.log('ffmpeg exited with code ' + code);
});

ffmpeg.stderr.on('data', function (data) {
    // console.log('stderr: ' + data);
});

io.on('connection', (socket) => {
    console.log('A user connected');

    ffmpeg.stdout.on('data', function (data) {
        var frame = new Buffer(data).toString('base64');
        io.sockets.emit('canvas',frame);
    });
    
    socket.on('message', (msg) => {
        switch (msg) {
            case 'client_event' :
                console.log("Client Events: " + msg);
                disconnect_client();
                break;
        }
    });

    socket.on('disconnect', () => {
        console.log("Disconnected from server");
    });

    socket.on('forceDisconnect', function() {
        socket.disconnect();
    });
}); 

http.listen(port, () => {
    console.log('listening on localhost:' + port);
});

• On the client Side receive the data from socket and show it on a canvas continuously.

    try {
        var canvas = document.getElementById('videostream');
        var context = canvas.getContext('2d');
        socket = io(null, { transports: ['websocket']});
        socket.on('canvas', function(data) {
        var imageObj = new Image();
        imageObj.src = "data:image/jpeg;base64,"+data;
        imageObj.onload = function(){
            context.height = imageObj.height;
            context.width = imageObj.width;                      
            context.drawImage(imageObj,0,0,context.width,context.height);
        }
        });
    } catch(e){ }

   
<canvas class="shadow" id="videostream" width="640" height="480"></canvas>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.7.2/socket.io.js"></script>

You may take a look at my following project on Github.

https://github.com/joynalabedinparag/Broadcast-webcam-ffmpeg

This solution worked on linux. Windows user may take a look at this post about how to capture webcam in windows using Ffmpeg.

Abel