ffmpeg audio - video sync issue (audio ahead of video) - while screen recording using x11grab

Question

While screen-recording using the below ffmpeg options I consistently get audio ahead of video, delay is in the order of few seconds

ffmpeg cmd:

ffmpeg -y -f x11grab -thread_queue_size 1024 -draw_mouse 0 -video_size 1920x1080 -i :0 -f pulse -thread_queue_size 1024 -i default -c:v libx264 -threads 0 -preset faster -c:a flac -async 1 -vsync 1 -crf 30 -crf_max 33 -f matroska output.mkv

ffprobe output below:

$ ffprobe demo.mkv 
ffprobe version 3.4.4-1~16.04.york0 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from 'demo.mkv':
  Metadata:
    ENCODER         : Lavf57.71.100
  Duration: 01:00:31.93, start: 0.000000, bitrate: 416 kb/s
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080, 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
    Metadata:
      ENCODER         : Lavc57.89.100 libx264
      DURATION        : 01:00:31.928000000
    Stream #0:1: Audio: flac, 48000 Hz, stereo, s16 (default)
    Metadata:
      ENCODER         : Lavc57.89.100 flac
      DURATION        : 01:00:30.912000000

I have even tried using avoid_negative_ts flag, but it didnt help with the sync issue.

OS: Ubuntu-18.04

ffmpeg version: 3.4

Update: (with ffmpeg-4.0) I have since split this process into 2 parts and tried as shown below (quality is better, but audio/video sync is still an issue)

# STEP-1: screen recording
/usr/bin/ffmpeg -y -f x11grab -thread_queue_size 1024 -draw_mouse 0 \
-video_size 1920x1080 -i :91141925 -f pulse -thread_queue_size 1024 \
-i virtual_sink.monitor -c:v libx264rgb -threads 0 -preset ultrafast \
-c:a flac -ac 1 -crf 0 -f matroska output.mkv

# STEP-2: audio/video encoding
/usr/bin/ffmpeg -y -i output.mkv -c:v libx264 -threads 0 \
-preset faster -pix_fmt yuv420p -c:a copy -ac 1 -crf 25 \
-f matroska final_output.mkv

STEP-1 uses much less CPU during screen recording, but audio is still ahead of video. the display I am using is xvfb one in STEP-1 (since, this is a headless machine in the cloud)

Also, i have tried flags -filter_complex aresample=44100 -vsync 1 in STEP-1 to no avail.

Can someone please help !

Update 2: (with latest ffmpeg from git master)

IT WORKED ! THANKS @llogan for the helpful comments.

Lordly

Your ffmpeg is old. First step is to try a recent version from the git master branch. You can compile or download (although I'm not sure if the static build has pulse support). – Roughen

i have tried with latest ffmpeg version 4.0 and still the same issue. – Lordly

I meant to use a build from the git master branch, not a release version. – Roughen

thanks @Roughen , that worked, used the latest ffmpeg & x264 version from the git master as suggested. – Lordly

Answer 1

I had the same issue (although in my case the video was ahead of the audio). Just switching -f pulse for -f alsa, as suggested on the Archlinux forum, removed the delay.

Falchion

This will record from microphone, not from desktop. – Canaanite

@Canaanite I’m not sure it’s that simple. I presume the default device is used if the device is not explicitely set, and I guess the default device may be different between Alsa and PulseAudio, hence the behaviour you observe ? – Falchion

I have the same issue as that guy from the linked forum. If I set -i default it always captures sounds from the microphone, no matter if I used -f pulse or -f alsa. If I want to capture desktop sound with Pulse I need to point the monitor device with -i option. With -f alsa I need to create a loopback virtual device and make a special config file for Alsa. trac.ffmpeg.org/wiki/Capture/ALSA#Recordaudiofromanapplication – Canaanite

Answer 2

After updating to Mint 21 (aka Ubuntu 22.04) ffmpeg capture with x11grab using -f pulse results in audio out-of-sync with video by a few seconds. I tried some suggestions, like using -copyts, -async 1, -itsoffset with different values. None corrected the issue.

There is apparently a bug causing this problem, and it reads like it is 'too old to be corrected' (https://trac.ffmpeg.org/ticket/10114). While updating ffmpeg to V5 or 6 may solve the issue, a simple download for these versions is not available on Mint 21/Ubuntu 22.04. There are several PPA's that claim to install V5 or 6, but they don't work due to dependency issues. The other option is to compile from source. I decided against this after reading the compilation guide.

Changing -f pulse to -f alsa as suggested in another answer corrects the audio sync, but then ffmpeg doesn't understand the pulse sources, so you have to change the way you specify the audio source. Although '-i default' works, it may not pick the correct source. I first tried to specify the internal sound card from the 'arecord -L' list (eg. -i hw:CARD=PCH,DEV=0), but that resulted in silent audio.

To find which source to use, use the pacmd cmd:

pacmd list-sources|grep name:|sed -r 's|name:\s<(.*)>|\1|'
##result##
    alsa_input.usb-MACROSILICON_2109-02.analog-stereo
    alsa_output.pci-0000_00_1b.0.analog-stereo.monitor

In my case the second line was the source I needed. To create an ffmpeg-usable name, create a .asoundrc file in your home directory that maps the alsa name to the pulse source. After you do this, you'll need to reboot or restart alsa and pulse:

nano ~/.asoundrc
pcm.internal_monitor {
  type pulse
  device alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
}

ctl.internal_monitor {
  type pulse
  device alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
}

If you've restarted the sound system or pc correctly, your new entry (internal_monitor) will appear in the alsa list:

arecord -L|grep internal
##result##
internal_monitor

Now use ffmpeg with -i internal_monitor to select your alsa sound source. After the change, only two entries changed in the command:

-f pulse 
##changed to##
-f alsa

-i alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
##changed to##
-i internal_monitor

The new alsa-friendly with audio-in-sync command is:

ffmpeg -f alsa -thread_queue_size 1024 \
 -i internal_monitor -f x11grab -thread_queue_size 1024 \
 -r 30 -s 1280x720 -i :0.0+0,152 \
 -acodec aac \
 -vcodec libx264 -preset ultrafast -crf 18 \
 -vf format=yuv420p -profile:v main \
 -f mpegts /home/ken/Videos/test.mkv -y

Icsh