Recording cross-platform (H.264?) videos using WebRTC MediaRecorder

Question

I have the following specified with my MediaRecorder implementation:

const getMediaRecorderOptions = function () {
    var options = { mimeType: "video/webm;codecs=vp8" }; // 9 was lagggy, cpu-intensive

    if (!MediaRecorder.isTypeSupported(options.mimeType)) {
        logger.recorderLog(options.mimeType + " is not Supported");
        options = { mimeType: "video/webm;codecs=vp8" };

        if (!MediaRecorder.isTypeSupported(options.mimeType)) {
            logger.recorderLog(options.mimeType + " is not Supported");
            options = { mimeType: "video/webm" };

            if (!MediaRecorder.isTypeSupported(options.mimeType)) {
                logger.recorderLog(options.mimeType + " is not Supported");
                options = { mimeType: "" };
            }
        }
    }
    return options;
}

Obviously, this is just for webm which isn't supported on iOS Safari or MacOS. I'm trying to avoid doubling our storage and introducing encoding complexity. Is there any way MediaRecorder on Chrome can record directly to a cross-platform container format, from any platform?

Indispose

Answer 1

You should be able to record to webm/h.264

var options = {mimeType: 'video/webm;codecs=h264'};

media_recorder = new MediaRecorder(stream, options);

So you have the right cross platform video format (H.264) in a WebM container.

Now you could try ffmpeg.js and just change the container from WebM to mp4 - coping the H.264 stream - no transcoding (-vcodec copy).

I recorded to webm/h.264 in Chrome but I didn't try re-wrapping it with ffmpeg.js.

Eraste

Neat - Would I be able to play those on iOS / macOS? I was under the impression that no webM container would play in Safari/Chrome iOS. – Indispose

I doubt WebM/H.264 plays in Safari but theoretically - since I haven't tried it - you can try to change the container from WebM/H.264 to MP4/H.264 using ffmpeg.js straight inside the recording browser as part of the recording process. Since you are not transcoding - the video stays H.264 - it should be a relatively lightweight operation. – Eraste

I am working on this for a long time and still trying to figure it out. the Chrome Android which comes with Android Device, like Huawei P9, won't have H.264 although it passes the test above. it won't record in H.264. Safari won't know WebM, it just knows MP4/mpeg with H264. Also, the Samsung S9 non-default Browser passes the test and can record H.264 using WebM container only. – Vietcong

-vcodec copying the h264 webm to mp4 worked like a charm for me in the browser (tested in chrome), performance is perfect with videoconverter.js – Anjanetteanjela

This worked for me. However any idea why the browser stops playing the video if I explicitly define the codec? E.g. <source src="video.webm" type={'video/webm; codecs="avc1.4D401E, mp4a.40.2"'} /> – Blunk

@Blunk I wish there is just "the browser" playing video. You'll deal with browsers, browser versions, OSs and OS versions. Your codec string may be wrong or doesn't match the video/audio stream. – Eraste

@MarkusSchumann you are right. How would I know the exact codec created by the vidoe/audio stream when all we provided to MediaRecorder is video/webm;codecs=h264? – Blunk

@Blunk - you have to parse the recorded video find SPS/PPS and assemble the correct codec string. Since your video is playing just fine without it - I am not sure what you are trying to accomplish. – Eraste

@MarkusSchumann the video is not playing fine without it on certain browsers like Facebook's in-app browser with user agent FBAN/FBAV. How would you parse the video to find SPS/PPS and assemble the correct codec? – Blunk

@Blunk - why don't you ask a new question in the H.264 tag and I'll take a look. – Eraste