mp3 support on Firefox MediaSourceExtension

Question

I'm looking into implementing adaptive and progressive audio streaming in the browser, with no plugins. MSE is the HTML5 API I was waiting for, available in FF 42, but it seems that the audio format support in Firefox is not there?... mp3 audio is not working when using the MSE APIs.

Here's a code snippet:

var mediaSource = new window.MediaSource();
var audioSourceBuffer;

mediaSource.addEventListener('sourceopen', function (e) {
    try {
        var mimeType = "audio/mpeg";
        audioSourceBuffer = mediaSource.addSourceBuffer(mimeType);
    } catch (e) {
        log('Exception calling addSourceBuffer', e);
        return;
    }
}

I get a NotSupportedError exception when calling addSourceBuffer.

Doesn't Firefox support mp3? from MDN list of supported formats (https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats) it implies that mp3 support should be there if the OS supports it - and the OS I am testing on (OSX) does support it.

Any help appreciated!

Monthly

Answer 1

OK, so I was able to figure it out with the kind help of Mozilla engineers working on Media Source Extension.

The first key thing to note on MSE (Media Source Extension) is that it does not necessarily support all media formats supported by the browser's Audio Element. To illustrate this, although Firefox will play mp3 file when fed directly to the browser or directly to an Audio element, it will not play the same mp3 file if you feed it into a media source buffer.

Now, which media format are actually supported by Firefox' MSE implementation? the answer is that as of Firefox 42 only fMP4 (Fragmented MP4) is supported by default. webm is also supported, but not by default and your users will have to turn it on manually through Firefox' about:config page. The fMP4 mimeType to feed the Media Source object when creating a new buffer is: audio/mp4; codecs="mp4a.40.2"

And if you're wondering what the heck is fMP4, it is a standard which is part of the MPEG-4 standard, more specifically part 12: "ISO Base Media File Format (ISOBMFF) using non-multiplexed audio/video". Look it up if you're interested in more details.

From my experience fMP4 is supported across all major browsers and OS - which makes fMP4 a good format candidate for adaptive and progressive streaming.

HTH!

Monthly

If you have the whole buffer, you can set the source of your audio element to a blob URL: player.src = URL.createObjectURL(new Blob([buffer], { type: 'audio/mpeg' })) – Dieppe

Have you been able to implement this with fMP4? I struggle to get my head around this format. See #64433922 – Prisage

Answer 2

i wrote something to encapsulate mp3 inside an mp4 on the fly in javascript, specifically to get this very thing working in firefox. i only needed it for an 128kbps stream, so this only works for that specific bitrate, with no album art. in case anyone else might find this useful: https://gist.github.com/fanfare/0fa525af28b275fd6623942d7e9d70dd

Ribal

Answer 3

I use FF42 on linux, and, contrary to what I would believe reading the documentation on MDN, MediaSource API is not enabled by default.

Have you tried to go in about:config and tweaking the parameters dealing with the codecs supported by the MediaSource API? I have switches for MP4 and webm. Not sure it would help for MP3, but it may be worth giving it a try.

Casanova

So it appears that MSE is enabled by default from FF 42, and supports mp4 and webm. See Firefox about:config and filter by mediasource. However, I have tried with mimeType = "audio/mp4" and while I do not get a NotSupportedException (as I got with mimeTyle = "audio/mpeg") - still firefox is not playing even when I feed it m4a files (Audio MP4). Any thoughts? – Monthly

Just to be sure, if you type the full path of the m4a file in the url field, does FF play the sound? – Casanova

After the update to Firefox 43 on linux, MediaSource API is now enabled by default. However, I have the same error than yours, with mimeType = "audio/mpeg" or "audio/mp3". Funny thing is that I can still directly play the mp3 when I type the url in the urlbar. – Casanova

Same error with a ogg file. So the problem is not due to licensing restrictions. It's maybe the syntax of mimeType which is wrong. – Casanova

The only two media types supported for Media Source on firefox are mp4 (on by default) and webm (off by default). – Monthly

BTW - the mime type for mp4 on firefox is: "audio/mp4" and for most of the fMP4 (Fragmented MP4) the codec is "mp4a.40.2" which will make the overall mime type string fed into the mediaSource object "audio/mp4; codecs="mp4a.40.2"". – Monthly