Live streaming through MP4

Question

I am working on an online TV service. One of the goals is for the video to be played without any additional browser plug-ins (except for Flash).

I decided to use MP4, because it is supported by the majority of HTML5 browsers and by Flash (for fallback). The videos are transcoded from ASF on a server by FFMpeg.

However, I found that MP4 cannot be live-streamed because it has a moov atom for metadata that has to specify the length. FFMpeg cannot directly stream mp4 to stdout, because it puts the moov at the end of the file. ( Live transcoding and streaming of MP4 works in Android but fails in Flash player with NetStream.Play.FileStructureInvalid error )

Of course, MPEG-TS exists, but it is not supported by HTML5 <video>.

What I thought about is a method to transcode the stream in real-time to MP4, and on each new HTTP request for it, first send a moov that specifies a very long number for the video's length, and then start sending the rest of the MP4 file.

Is it possible to use MP4 for streaming that way?

After some research and av501's answer, I understand that the sizes of the frames must be known so that it can work.

Can the mp4 file be segmented into smaller parts so that it can be streamed?

Of course, switching to another container/format is an option, but the only format compatible with both Flash and HTML5 is mp4/h264, so if I have to support both, I'd have to transcode twice.

Lodicule

Answer 1

Here's my thoughts guys some of it might be right on others way way off. I plead ignorance because no one have really documented this process fully, its all an educated guess.

AvAssetWriter only encodes to a file, there seems to be no way to get encoded video to memory. Reading the file while it is being written to from a background thread to say a socket results in an elementary stream, this is essentially an m4v, which its a container with h264/acc mdata, but no moov atoms. (in other words no header) No apple supplied player can play this stream, but a modified player based on ffplay should be able to decode and play the stream. This should work, because ffplay use libavformat which can decode elementary streams, one caveat since there is no file length info, some things have to be determined by the play, the DTS and PTS and also the player can't seek within the file.

Alternatively an the raw naul's from the m4v stream can be used to construct an rtmp stream.

If you want to discuss further you can contact me directly.

How you get at the data.

Since your going to have to rebuild the file on the receiving side anyway, I guess you could just kind of segment it, Steve Mcfarin wrote a little appleSegmentedEcorder you can find on his github page, this solves some of the issues for moov atoms since you have all the file info.

Colliery

here's a good explanation, starting to sound like rtmp or some custom protocol is the way to go fabiensanglard.net/mobile_progressive_playback/index.php – Colliery

your answer has caveat , it has written on the above. Time is the matter for streaming and playback. so -1 – Comstock

Answer 2

You may use fragmented MP4. A fragmented MP4 file is built a follows:

moov [moof mdat]+

The moov box then only contains basic information about the tracks (how many, their type , codec initialization and so on) but no information about the samples in the track. The information about sample locations and sample sizes is in the moof box, each moof box is followed by a mdat that contains the samples as described in the preceding moof box. Typically one would choose the length of a (moof, mdat)-pair to be around 2,4 or 8 seconds (there is no specification on that but these values seem to be reasonable for most usecases).

This is a way to construct a neverending MP4 stream.

Meet

have a look here: groups.google.com/forum/?fromgroups=#!topic/… this guy seems to have a very similar problems. – Meet

Ultimately fragmented mp4 is like Apple HTTP live streaming and does a good job. I disregarded that in the beginning because I thought that fragmenting seemed to be an unnecessary complication, but now I see it's the only option. – Lodicule

@SebastianAnnies This seems to work, however (at least) Safari seems to request each fragment with a separate Range GET request (from fragment start until EOF, but terminates prematurely), making the whole handling of the stream much more of a nuisance. Do you have the same experience? – Benson

Answer 3

No, it is not just the very long length.. you need to know exact size of every frame to create the header in a mp4. [which is why it gets created at the end by the various encoders].

Brisling

Can't that info just be left empty? After some tweaking, I managed to create an m4v file (basically an MP4) without an moov atom, and it played in Totem, without showing any info about video length. – Lodicule

Is your m4v a mp4 (container) containing the elementary file or just the elementary file? Without the header it is no longer compliant. So whether it plays or not depends on how intelligent the player is. Some players may work hard and try to fix the file. But most will not play it. – Brisling

Yes, it seems that m4v can be either just a mp4 file or a raw h264 stream. The ones which worked in the browser were mp4, and the one generated by ffmpeg was a raw stream. So it seems that it can't be done after all. – Lodicule

Answer 4

Just looking at 2nd para of your question ("The videos are transcoded from ASF on a server by ffmpeg."), you mentioned you are using ffmpeg to transcode videos on server.

Use qt-faststart or MP4Box to place MOOV atom in the beginning of the file. (also you make sure that using H264 Video & AAC Audio codec for universal support)

Hope this helped you.

Duckbill

It doesn't. After numerous google searches, I found that qt-faststart was mentioned a lot - but it won't work for live streaming since the stream is CONTINUOUS - the frames/length cannot be known from the start, therefore an MOOV atom cannot be created. qt-faststart would help if what I needed was pseudostreaming (continuous download). – Lodicule