Improve bash script for checking when ffmpeg hangs streaming IP cam rtsp to youtube

Question

Background

I have a garden IP cam id like to stream live to youtube, so i decided to use ffmpeg to achieve this.

The problem

Whenever my IP cam restarts or loses connection; ffmpeg will get stuck on the same frame and not resume once the IP cam is back online.

My solution

I have the ffmpeg output logged to a file, then have a script fetch the last line of the log every few seconds and compares frame numbers. If the frame numbers match, it kills ffmpeg process and starts another ffmpeg process.

My question

Is there a better way more efficient way?

logchecker.sh

#/bin/bash
while true
do
    frameA=$(tail /home/daniel/output.txt -n 1 | sed -nr 's/.*frame=(.*)fps.*/\1/p')
    echo "$frameA"
    sleep 3
    frameB=$(tail /home/daniel/output.txt -n 1 | sed -nr 's/.*frame=(.*)fps.*/\1/p')
    echo "$frameB"

    if [ "$frameA" = "$frameB" ]
    then
        echo "Camera has hung"
        pkill ffmpeg
        echo "killed ffmpeg..."
        echo "Waiting 30 secs"
        sleep 30
        bash /home/daniel/ffmpeg.sh &
        echo "started ffpmeg.."
        echo "Waiting 30 secs"
        sleep 30
    else 
        echo "proceed"
    fi

    sleep 2
done

ffmpeg.sh

#!bin/bash
sleep 30
ffmpeg -f lavfi -i anullsrc -rtsp_transport udp -i rtsp://user:password@url:5544/live0.264 -bufsize 5000k -c:v copy -c:a mp3 -b:a 1 -f flv rtmp://a.rtmp.youtube.com/live2/xxxx-xxxx-xxxx-xxxx 2> /home/daniel/output.txt

Sophi

Answer 1

I've been trying to do something similar. I use ffmpeg to record IP Camera footage (Reolink) since the built-in ftp feature skips recording a few seconds between each file. And sometimes the recordings disconnects for 12 seconds. I use Raspberry Pi as a server and the command I've been using to record the camera is as follow:

pgrep -x ffmpeg || ffmpeg -stimeout 5000000 -rtsp_transport tcp -i rtsp://user:[email protected]/h264Preview_01_main -c copy -map 0 -f segment -segment_time 60 -segment_format mp4 -segment_atclocktime 1 -reset_timestamps 1 -strftime 1 /srv/dev-disk-by-id-usb-Seagate_Expansion_Desk_2HC015KJ-0-0-part1/camera/Backyard-%Y%m%d-%H%M%S.mp4

The command I've been using to check if ffmpeg hangs is:

test `stat -L -c %Y /proc/$(pgrep -x ffmpeg)/fd/4` -ge $(($EPOCHSECONDS - 10)) || killall ffmpeg

Turns out pipe number 4 is a syslink to the current file being written, so I check for last modified time and if it's not within the last 10 seconds then I kill the process. Time will tell if it's the correct solution.

Tern

Very nice, thank you! Just a note, with stat -L -c %y /proc/$(pgrep -x ffmpeg)/fd/4 | cut -d ':' -f 3 | cut -d ' ' -f 1 you can also have sub-second resolution. – Frontal