Can YUV -> RGB conversion be hardware accelerated?

Question

We have an application that reads a GigE YUV video stream and displays it on the screen. By profiling, we have learned that the function converting each frame from YUV (UYVY) to RGB24 is taking at least an order of magnitude more time and CPU than any other piece of our camera-to-screen pipeline.

The conversion function we are using is provided by the GigE software vendor (Pleora) and is slightly faster than our own 'naive' (non-optimized) implementation. We are using DirectShow for the rest of our pipeline. 'Task-manager benchmarking' shows for our 1080p 30fps stream, a cpu usage of 4-5% when we skip the conversion (and get garbled image of course), and 15-19% CPU usage when we call the conversion function.

The questions we have are:

1. Is there a DirectShow filter that will do this conversion for us, hopefully in a more performant manner, rather than relying on a 3rd party SDK or our own (CPU-based, serial) conversion function ?
2. Must this conversion be done on the CPU, or is it somehow able to be offloaded to the GPU for parallel processing?

Thanks! Eric.

Ie

The cost comes from reading and writing every byte in the image thus saturating the memory bandwidth. GPU processing is only beneficial for high computational overhead to low bandwidth ratios. This is one of the benefits for video cards taking YUV overlays. – Broca

Answer 1

The conversion is perhaps a good candidate for GPU processing, however what are you going to do with the converted data? If you need it for further processing in software, then reading back from video adapter might ruin all the gain you might have obtained by offloading processing to GPU. If you need it for presentation purposes only, then you don't need to convert, you can deliver YUV image right to video adapter and let it present it this way (which is the ideal configuration of the pipeline, since you don't have any conversion at all).

Talking about software conversion, I am not sure about the quality of the conversions you are using right now, however there are highly optimized (SIMD-enabled) conversions available:

1. Standard Windows Vista+ DMO
2. FFmpeg's libswscale
3. Intel IPP Primitives

All three are more or less easily pluggable into DirectShow pipeline. Additionally high resolution images are good candidates for parallel software processing too.

Applied

Thanks! Excellent and clear reply. The final output will be to a WPF D3DImage, which as I understand it has to be a Direct3D RGB32 surface, so asking the video card to display it as YUV won't be an option. I'll take a look at the three libs you recommend and take that as the best possible solution. – Ie