I'm experimenting with the new MediaSource API available in Chrome.
I'm trying to append binary data on the fly from a WebSocket to the video media source.
Starting with the example at https://html5-demos.appspot.com/static/media-source.html, my code is currently:
var websocket = new WebSocket('ws://localhost:8080');
websocket.binaryType = 'arraybuffer';
var mediaSource = new MediaSource();
var buffer;
var queue = [];
var video = $('.video')[0];
video.src = window.URL.createObjectURL(mediaSource);
mediaSource.addEventListener('sourceopen', function(e) {
video.play();
buffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.64001E"');
buffer.addEventListener('updatestart', function(e) { console.log('updatestart: ' + mediaSource.readyState); });
buffer.addEventListener('update', function(e) { console.log('update: ' + mediaSource.readyState); });
buffer.addEventListener('updateend', function(e) { console.log('updateend: ' + mediaSource.readyState); });
buffer.addEventListener('error', function(e) { console.log('error: ' + mediaSource.readyState); });
buffer.addEventListener('abort', function(e) { console.log('abort: ' + mediaSource.readyState); });
buffer.addEventListener('update', function() { // Note: Have tried 'updateend'
if (queue.length > 0 && !buffer.updating) {
buffer.appendBuffer(queue.shift());
}
});
}, false);
mediaSource.addEventListener('sourceopen', function(e) { console.log('sourceopen: ' + mediaSource.readyState); });
mediaSource.addEventListener('sourceended', function(e) { console.log('sourceended: ' + mediaSource.readyState); });
mediaSource.addEventListener('sourceclose', function(e) { console.log('sourceclose: ' + mediaSource.readyState); });
mediaSource.addEventListener('error', function(e) { console.log('error: ' + mediaSource.readyState); });
websocket.addEventListener('message', function(e) {
if (typeof e.data !== 'string') {
if (buffer.updating || queue.length > 0) {
queue.push(e.data);
} else {
buffer.appendBuffer(e.data);
}
}
}, false);
I consistently get the error message: InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source. after one append. It looks like the MediaSource is closing immediately after the call to buffer.appendData().
Any way to do this elegantly?
Note: chrome://media-internals/ doesn't return any useful information.
$('video)<-- missing ' – Aquiculture'updateend'instead of'update', and add some logs for'error','sourceended'and'sourceclose'events so you know when it's failing. If it's right after the very first.appendBuffer(), it might be something with the video file itself. – Corpuz'error'is on thebuffer, the source ones are on themediaSource. See the events for each listed on the w3 site – Corpuzwebsocketand themediaSource, and you might be trying to append before it's open or appending out of order (though I'm still not clear if that's ok or not for mediaSource). You can checkif (buffer.updating || mediaSource.readyState != "open" || queue.length > 0)to make sure it's not updating, is open, and doesn't already have a queue lined up before appending directly from the websocket. Also, if possible, be sure you're using a video that you know works with mediaSource, like this one. – Corpuzqueue.length > 0seems to have helped. Also, it looks like only H264-DASH is support - so I've switch for VP8 and am seeing some frames now. Getting anabortmessage now consistently after 5-6 frames though. (updatestart, abort, updateend, sourceclose). – CharbonnierMedia segment did not begin with keyframe.in chrome://media-internals/ which I think is causing this secondary issue. Have updated my question with all the latest code/details. – Charbonnierqueue.length > 0and pointing out that the file may not be MediaSource compatible. – Charbonnier