What ffmpeg settings to ensure 0 duplicate frames and 0 dropped frames when capturing to mpeg-2 program stream using ffmpeg/avfoundation on Mac?

Question

I'm trying to capture to a DVD compliant mpeg-2 file (ffmpeg: -target ntsc-dvd) from the HDMI output of a camcorder into a Magewell HDMI to USB 3.0 box into my Late 2012 15" non-retina MacBook Pro (quad core 2.3, 16gb ram, ssd), using ffmpeg/avfoundation.

I've tried everything I can think of, or find online. I'm still getting duplicate and dropped frames, which either leads to audio/video sync issues, or audio dropouts, especially for longer recordings. I need this to be stable for recordings of up to 2.5 hours. This is the Terminal output for a 1.5 hour recording:

Lapaki:~ Lapaki$ /Users/Lapaki/Desktop/ffmpeg -f avfoundation -video_size 960x540 -pixel_format uyvy422 -framerate ntsc -i "XI:XI" -vf crop=iw-240:ih:120:0 -target ntsc-dvd -aspect 4:3 -q:v 3 -ab 256k /Users/Lapaki/Desktop/FF\ Test/`date +%F`\ `date +%H_%M_%S`.mpg
ffmpeg version 3.2.3-tessus Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzmq --enable-version3 --disable-ffplay --disable-indev=qtkit --disable-indev=x11grab_xcb
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, avfoundation, from 'XI:XI':
  Duration: N/A, start: 610606.984208, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 960x540, 29.97 fps, 29.97 tbr, 1000k tbn, 1000k tbc
    Stream #0:1: Audio: pcm_f32le, 48000 Hz, stereo, flt, 3072 kb/s
Output #0, dvd, to '/Users/Lapaki/Desktop/FF Test/2017-02-15 17_46_28.mpg':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 720x480 [SAR 8:9 DAR 4:3], q=2-31, 6000 kb/s, 29.97 fps, 90k tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc57.64.101 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 9000000/0/6000000 buffer size: 1835008 vbv_delay: -1
    Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
    Metadata:
      encoder         : Lavc57.64.101 ac3
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (pcm_f32le (native) -> ac3 (native))
Press [q] to stop, [?] for help
[swscaler @ 0x7fd315892800] Warning: data is not aligned! This can lead to a speedloss
frame=   20 fps=0.0 q=3.0 size=     298kB time=00:00:00.65 bitrate=3721.4kbits/sframe=   35 fps= 35 q=3.0 size=     498kB time=00:00:01.13 bitrate=3591.2kbits/sframe=   50 fps= 33 q=3.0 size=     708kB time=00:00:01.64 bitrate=3519.4kbits/sframe=   65 fps= 32 q=3.0 size=     920kB time=00:00:02.16  
...
 bitrate=2721.7kbits/frame=162094 fps= 30 q=3.0 size= 1796936kB time=01:30:08.47 bitrate=2721.7kbits/frame=162109 fps= 30 q=3.0 size= 1797142kB time=01:30:08.98 bitrate=2721.8kbits/frame=162110 fps= 30 q=3.0 Lsize= 1797202kB time=01:30:09.01 bitrate=2721.9kbits/s dup=221 drop=0 speed=   1x    
video:1579050kB audio:168069kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.866632%

I deleted the middle part (denoted by the "..."), which is just a lot more of the same accumulating information. By the end there are 221 duplicate frames, and for this one I didn't get any dropped frames, but that happens every other time as well it seems.

With this code, the audio seems to stay in pretty good sync, but I get little dropouts every 30 seconds to a minute or so. On this recording, there are dropouts at:

00:00:43, 00:01:19, 00:01:47, 00:02:17, 00:03:18, ...

I stopped listening there; they happen at the end too, so I assume similarly spaced dropouts are happening throughout the file.

Is there some secret ffmpeg code to ensure there are no dropped or duplicate frames when capturing from a live source to dvd compliant mpeg-2 files?

When I convert from pretty much any type of file to mpeg-2 files using -target ntsc-dvd, the speed is something like 10x on this machine, so it seems like it would have no problem keeping up with a live source, right?

I've also tried constant bit rate using -b:v 5000k -minrate 5000k -maxrate 5000k -bufsize 2500k, which also doesn't prevent dropped/duplicate frames.

I've tried separating out the audio and video inputs, which doesn't solve it.

I've tried using -vsync 0 on the video input, which does seem to solve the issue, because the output doesn't report dups/drops, but the audio/video drift out of sync more and more, so that hasn't worked either.

Thanks so much for any help. I've been testing and testing and searching and searching for weeks...

Hermaherman

You'll need -vsync 0 to avoid drop/dups if the feed isn't stable. To try to rectify A/V sync, add -af aresample=async=1 after the crop and check. – While

Thanks, @Mulvya! Does the order of the aresample=async=1 matter? Why after the crop? vsync=0, does prevent dropped frames, but then the I get this: ... frame=77446 fps= 30 q=3.0 size= 1007172kB time=00:43:04.38 bitrate=3192.5kbits/s[mpeg2video @ 0x7fc54402cc00] Invalid pts (77468) <= last (77468) Video encoding failed Conversion failed! How can I tell that the feed isn't stable? Is there some other output that I can look at to monitor capture related stats? – Hermaherman

@Mulvya, just tried this ffmpeg -f avfoundation -video_size 960x540 -pixel_format uyvy422 -framerate ntsc -thread_queue_size 12B -vsync 0 -i "XI:XI" -vf crop=iw-240:ih:120:0 -af aresample=async=1 -pix_fmt yuv420p -aspect 4:3 -s 720x480 -q:v 3 -maxrate 5000k -bufsize 2000k -acodec ac3 -ac 2 -ab 256k -ar 48000 -f dvd test.mpg and got this frame=89093 fps= 30 q=3.0 size= 581436kB time=00:49:32.70 bitrate=1602.3kbits/s[mpeg2video @ 0x7fb5d206bc00] Invalid pts (89101) <= last (89101) Video encoding failed Conversion failed!. Need to capture 2-3hr long clips, and sync is still off. Ideas? – Hermaherman

@Mulvya, should -vsync 0 be placed before the input or after? – Hermaherman

After the input. Did you try the TS with -copyts? – While

@Mulvya, I did try -copyts, also with start_at_zero. Where should I put the -copyts, after or before the input? – Hermaherman

@Mulvya, I just moved -vsync 0 after the input, and then tried -copyts before and after the input, in both instances I get buffer underflow errors and the bitrate and fps stay at 0: [dvd @ 0x7f8ef8854800] buffer underflow st=0 bufi=17000 size=18868 frame= 20 fps=0.0 q=1.6 size= 270kB time=168:06:58.35 bitrate= 0.0kbits/. – Hermaherman

@Mulvya, does it matter where -vsync 0 is in relation to -aresample=async=1 or the crop filter? – Hermaherman

No. Both should be after inputs and before outputs. Try with -copyts before -i XI:XI; remove -f dvd and output to test.ts. If there's an error, add -avoid_negative_ts make_zero. Don't add -start_at_zero – While

To be clear, add -avoid_negative_ts make_zero before input, right? also, should vsync 0 be before or after the crop filter? – Hermaherman

avoid.. goes after inputs, as does vsync. Doesn't matter where in relation to crop. – While

as a general rule for ffmpeg, apart from "before inputs" or "after inputs and before outputs", does order matter? – Hermaherman

Options can be input, output, or global. Input options come before that input. Output, before the output name and after inputs. Global, anywhere. Some options can be both input or output modifiying depending on placement. Generally, docs will indicate this. – While

@Mulvya, so -af aresample=async=1 does need to be after the crop filter though, per your first comment? So does order of filters matter? Filters can only be output options, correct? – Hermaherman

Actually no. I just said that so you would place it as an output option. I've had posters place it with abandon when not specified. Results in a waste of time. – While

Awesome, good to know :) – Hermaherman

Ok here is most recent code: ffmpeg -f avfoundation -video_size 960x540 -pixel_format uyvy422 -framerate ntsc -thread_queue_size 12B -copyts -i "XI:XI" -avoid_negative_ts make_zero -vf crop=iw-240:ih:120:0 -vsync 0 -af aresample=async=1 -pix_fmt yuv420p -aspect 4:3 -s 720x480 -q:v 3 -maxrate 5000k -bufsize 2000k -acodec ac3 -ac 2 -ab 256k -ar 48000 test.ts. I'm getting this output: frame=19550 fps= 30 q=3.0 Lsize= 171584kB time=211:58:49.98 bitrate= 1.8kbits/s speed=1.17e+03x. The resulting video seems to play fine, but the stats are quite strange and not reflective of the recording. – Hermaherman

I'm still getting sporadic audio dropouts as well... – Hermaherman

@Mulvya, any more ideas? The thing that is weird to me, is that if I run the same output options on a 1080p file, it converts at something like 3-4x the input framerate, and even slower/older machines can do the conversion at 2-3x, without any issues. Doesn't that mean that the problem has nothing to do with the speed of the computer? Could it be that the conversion is happening too fast, and the dropouts happen because FFmpeg is temporarily running out of video to convert? Could the -re switch or something similar help? Really baffled why we can't get this to work reliably... – Hermaherman

@Mulvya, at this point, I'd be happy to instead capture to a master/intermediate file of whatever type will be the most stable/reliable, and then start another ffmpeg process after maybe 5 seconds, to convert the master file to mpeg-2, using with the -re switch on the second process so that it could happen in parallel to the first process. Think that could be the answer? If so, what would you suggest as good output options for the master file? Assuming it would need to be a .ts container so that the second FFmpeg process could convert the first while it's being captured. Thougthts? – Hermaherman

@Mulvya, for the master file, I'm fine with it being fairly large if that helps with stability. I can just overwrite it on subsequent captures, or even add a tag-along process to delete it after the mpeg-2 conversion finishes. – Hermaherman