Reduce HLS latency from +30 seconds

Question

Ubuntu 12.04

nginx 1.2.4

avconv -version

avconv version 0.8.10-4:0.8.10-0ubuntu0.12.04.1, Copyright (c) 2000-2013 the Libav developers
  built on Feb  6 2014 20:56:59 with gcc 4.6.3
avconv 0.8.10-4:0.8.10-0ubuntu0.12.04.1
libavutil    51. 22. 2 / 51. 22. 2
libavcodec   53. 35. 0 / 53. 35. 0
libavformat  53. 21. 1 / 53. 21. 1
libavdevice  53.  2. 0 / 53.  2. 0
libavfilter   2. 15. 0 /  2. 15. 0
libswscale    2.  1. 0 /  2.  1. 0
libpostproc  52.  0. 0 / 52.  0. 0

I'm using avconv and nginx to create an HLS stream but right now my latency is regularly well over 30 seconds. After much reading I am aware that HLS has built in latency and that 10s is expected and even preferred but 30s seems quite extreme.

I've seen a lot of discussion on the nginx-rtmp google group, this thread in particular had a lot of suggestions. I have attempted to reduce solve my issue by reducing the hls_fragment and the hls_playlist_length but they haven't had a significant effect.

nginx.conf:

#user  nobody;
worker_processes  1;

error_log  logs/error.log debug;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    server {
        listen       8888;
        server_name  localhost;

        add_header 'Access-Control-Allow-Origin' "*";

        location /hls {
            types {
                application/vnd.apple.mpegurl m3u8;
                video/mp2t ts;
            }
            root /tmp;
        }

        # rtmp stat
        location /stat {
            rtmp_stat all;
            rtmp_stat_stylesheet stat.xsl;
        }
        location /stat.xsl {
            # you can move stat.xsl to a different location
            root /usr/build/nginx-rtmp-module;
        }

        # rtmp control
        location /control {
            rtmp_control all;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

rtmp {
    server {
        listen 1935;
        ping 30s;
        notify_method get;

        application myapp {
            live on;

            hls on;
            hls_path /tmp/hls;
            hls_base_url http://x.x.x.x:8888/hls/;
            hls_sync 2ms;
            hls_fragment 2s;

            #hls_variant _low BANDWIDTH=160000;
            #hls_variant _mid BANDWIDTH=320000;
            #hls_variant _hi BANDWIDTH=640000;
        }
    }
}

avconv command:

avconv -r 30 -y -f image2pipe -codec:v mjpeg -i - -f flv -codec:v libx264 -profile:v baseline -preset ultrafast -tune zerolatency -an -f flv rtmp://127.0.0.1:1935/myapp/mystream

Edit

I know that it's not avconv that is the bottleneck because I am also producing an RTMP stream using nginx. This RTMP stream has a very small delay (couple of seconds).

Jemy

As you have seen, the nature of HLS is high latency. 30 seconds isn't extreme... it's quite common. When YouTube does live events, delays of minutes are the norm (although they do have a much larger network to distribute to). The suggestions you have found are also correct. If they didn't make a difference in your latency, the problem might be on the encoding end. Take the HLS part out of the equation, and try to play back data from avconv/FFmpeg immediately. Measure the delay. It should be a second or less, but you never know. – Binocular

My current setup produces two streams, one which uses HLS, the other which uses RTMP, the RTMP stream has a very small delay in it (maybe a couple of seconds). – Jemy

@Jemy I see you're now using nodejs which might be interesting to use, but unfortunately I believe iOS requires that you use HLS for streaming or RTMP would have probably been my first choice. How is your latency now? – Impotent

@PanamaJack - Caveat, I have not been on this project for a couple of years. I do not believe that IOS was a problem, we were using nodejs to stream and then used a JS decoder library which wrote the frames to an HTML5 canvas. The latency was fine (couple of seconds) We couldn't use RTMP because that required a flash player in browser which won't give you access to the individual frames (which we needed). Note that any of this could have changed in the couple of years that I last worked on this. – Jemy

Answer 1

The answer detailing reducing the hls_playlist to 4s and the hls_fragment to 1s is good. However, it is important to note that if the source video has a key frame interval greater than the hls_fragment length then nginx will usually have to create longer fragments and the latency will increase.

The lowest latency we have achived was with the hls_fragment at 1s and the source video key frame interval also at 1s. With these settings we achieved latency of less than 7 seconds.

Bloodfin

I feel like this is more of a comment than an answer. – Kinny

I gave this as an answer because you absolutely must have the source key frame interval set to a low value otherwise you will not get latency below 10s as the origional question expected (and I am not allowed to add comments so how else could I make this important distinction that was not yet mentioned clear?) – Bloodfin

@Bloodfin how did you change ths ource video key frame interval? I'm trying to stream in the browser using hls.js and haven't found a way to accomplish this. – Precautious

@Precautious This needs to be set on whatever it is that generates your video. For testing you might try using the Larix Broadcaster app on a tablet of phone. Larix broadcaster allows you to change the video key frame interval: link – Bloodfin

Also possible when you use OBS or vmix. – Alcantar

Answer 2

I once had the same problem. I added two parameters in nginx.conf then the latency was cut to 10s even less. Here is my conf:

    application hls {
        hls_playlist_length 4s; 
        hls_fragment 1s
    }

However, I have to say 10s seems to be our limitation. Any better solutions?

Kindergarten

In the end we ended up dropping HLS entirely and switched to using a nodejs server which would pass the frames over a socket. link – Jemy