Using libav (ffmpeg), how to decode a video file directly to a GPU texture?

Question

I'm using ffmpeg's libav to decode video files on Mac. For supported codecs, it says it can use the Mac VideoToolbox framework to hardware-accelerate the decoding. Can I get the result of that decode directly as a Metal or CoreVideo buffer or texture, in GPU memory? My plan is to process it with compute shaders before sending it to the screen and I'd like to maximize framerate by removing CPU<->GPU transfers.

Is there an example of doing this anywhere?

Tailstock

Have you seen this? VT explanations begin at roughly 14:00 into the video, and later on it ties in with the use of Metal. – Newcomb

That's a good walkthrough of the Mac APIs, so it would be useful to someone doing MacOS-only work using only native MacOS, but unfortunately has no references to ffmpeg at all. – Tailstock

Have you ever checked vlc or mpv's source code? If I remember correctly, mpv's codec backend is thoroughly written in ffmpeg, and mpv implements no copy-back hardware decoding. So there should be some clues to solve your problem, I guess. – Teresiateresina

objc.io/issues/23-video/videotoolbox and developer.apple.com/library/archive/qa/qa1781/_index.html may be useful. I see that ffmpeg sets the CVPixelBuffer to be IOSurface backed, which I assume means that it's backed by gpu memory. – Plasticize

github.com/mpv-player/mpv/blob/… – Plasticize