HTML5 <audio> poor choice for LIVE streaming?

Question

As discussed in a previous question, I have built a prototype (using MVC Web API, NAudio and NAudio.Lame) that is streaming live low quality audio after converting it to mp3. The source stream is PCM: 8K, 16-bit, mono and I'm making use of html5's audio tag.

On both Chrome and IE11 there is a 15-34 second delay (high-latency) before audio is heard from the browser which, I'm told, is unacceptable for our end users. Ideally the latency would be no more than 5 seconds. The delay occurs even when using the preload="none" attribute within my audio tag.

Looking more closely at the issue, it appears as though both browsers will not start playing audio until they have received ~32K of audio data. With that in mind, I can affect the delay by changing Lame's MP3 'bitrate' setting. However, if I reduce the delay (by sending more data to the browser for the same length of audio), I will introduce audio drop-outs later.

Examples:

• If I use Lame's V0 encoding the delay is nearly 34 seconds which requires almost 0.5 MB of source audio.
• If I use Lame's ABR_32 encoding, I can reduce the delay to 10-15 seconds but I will experience pauses and drop-outs throughout the listening session.

Questions:

1. Any ideas how I can minimize the start-up delay (latency)?
2. Should I continue investigating various Lame 'presets' in hopes of picking the "right" one?
3. Could it be that MP3 is not the best format for live streaming?
4. Would switching to Ogg/Vorbis (or Ogg/OPUS) help?
5. Do we need to abandon HTML5's audio tag and use Flash or a java applet?

Thanks.

Biconcave

Answer 1

You can not reduce the delay, since you have no control on the browser code and buffering size. HTML5 specification does not enforce any constraint, so I don't see any reason why it would improve.

You can however implement a solution with webaudio API (it's quite simple), where you handle streaming yourself.

If you can split your MP3's chunk in fixed size (so that each MP3 chunks size is known beforehand, or at least, at receive time), then you can have a live streaming in 20 lines of code. The chunk size will be your latency.

The key is to use AudioContext::decodeAudioData.

// Fix up prefixing
window.AudioContext = window.AudioContext || window.webkitAudioContext;
var context = new AudioContext();
var offset = 0;
var byteOffset = 0;
var minDecodeSize = 16384; // This is your chunk size

var request = new XMLHttpRequest();
request.onprogress = function(evt)
{
   if (request.response)
   {
       var size = request.response.length - byteOffset;
       if (size < minDecodeSize) return;
       // In Chrome, XHR stream mode gives text, not ArrayBuffer.
       // If in Firefox, you can get an ArrayBuffer as is
       var buf;
       if (request.response instanceof ArrayBuffer)
           buf = request.response;
       else
       { 
           ab = new ArrayBuffer(size);
           buf = new Uint8Array(ab);
           for (var i = 0; i < size; i++) 
               buf[i] = request.response.charCodeAt(i + byteOffset) & 0xff;
       }
       byteOffset = request.response.length;
       context.decodeAudioData(ab, function(buffer) {
           playSound(buffer);
       }, onError);
   }
};
request.open('GET', url, true);
request.responseType = expectedType; // 'stream' in chrome, 'moz-chunked-arraybuffer' in firefox, 'ms-stream' in IE
request.overrideMimeType('text/plain; charset=x-user-defined');
request.send(null);

function playSound(buffer) {
    var source = context.createBufferSource(); // creates a sound source
    source.buffer = buffer;                    // tell the source which sound to play
    source.connect(context.destination);       // connect the source to the context's destination (the speakers)
    source.start(offset);                           // play the source now
                                           // note: on older systems, may have to use deprecated noteOn(time);
    offset += buffer.duration;
}

Truce

Thank you so much this worked perfectly for me. Interestingly, it does not work on mobile browsers - how can we modify it so that it does? I get the following error: Unhandled Promise Rejection: EncodingError: Decoding failed – Hooker