MediaRecorder changes size without provocation

Question

I'm using the MediaRecorder API along with the Canvas captureStream method to encode a VP8 video stream of a canvas in browser. This data is sent to FFmpeg via binary web socket.

var outputCaptureStream = $('canvas')[0].captureStream(30);
var mediaRecoder = new MediaRecoder(outputCaptureStream, {
  mimeType: 'video/webm'
});

mediaRecorder.ondataavailable = function (e) {
  ffmpegStdin.write(e.data);
}

mediaRecoder.start(1000);

For some reason, the stream seems to be randomly switching to a lower resolution mid-stream. FFmpeg isn't happy about this:

Input stream #0:0 frame changed from size:1280x720 fmt:yuv420p to size:1024x576 fmt:yuv420p

[vp8 @ 0x2a02c00] Upscaling is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. [vp8 @ 0x2a02c00] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])

I suspect that it has something to do with excessive CPU usage and that Firefox is trying to be helpful by scaling down the video. My questions:

• Does Firefox scale down the video on the fly?
• If so, what conditions cause this to happen? (CPU load? Stream backpressure?)
• Is it possible to prevent Firefox from doing this?
• Is there a different explanation for this behavior that I'm missing?

Dubose

is the canvas size fixed? are you streaming your camera stream into the canvas? – Derekderelict

@Derekderelict Canvas size is completely fixed. I'm drawing frames on requestAnimationFrame, capped at 30FPS via the "better refined approach" here: codetheory.in/… – Dubose

can you share the code, I could give it a try and see if I can spot the issue.. – Derekderelict

Answer 1

Firefox will rescale (downscale) WebRTC/getUserMedia video if it detects the system's CPU is being overloaded. There are a few prefs in about:config that control this behavior, but it's not controllable via JS.

You can disable the feature by setting

media.navigator.load_adapt=false

You can look at the other media.navigator.load_adapt.* flags for some control over the behavior. By default you will get downscaling if the CPU gets pegged more than 90% for 3 seconds.

Planchette

After further testing, I found that media.navigator.load_adapt=false has no bearing on whether or not the output video is dynamically scaled mid-stream. After more discussion in the IRC room, it was suggested that I open this bug report: bugzilla.mozilla.org/show_bug.cgi?id=1250345 – Dubose