How to display quick-updating images without large memory allocation?

Question

I've got a WPF application on an ultrasound machine that displays ultrasound images generated in C++ at a speed upwards of 30 frames per second.

From what I understand, the normal process for displaying images in WPF is to create a BitmapSource for your image and set the Source for your Image, which then causes it to invalidate and display.

Since BitmapSources do not implement IDisposable, using this method forced me to create 30 BitmapSources a second. For a 640x480 image with 32bppArgb format, this is around 30MB/sec of memory being allocated a second and then garbage disposed every 10 seconds, causing visible lag. Obviously not an acceptable solution.

My currently solution is:

In C++: I create a System.Drawing.Bitmap (WinForms bitmap) in Managed C++, do a memcpy from a pointer to populate the picture, use the Graphics object to do some additional drawing I need, and pass this into the C#/WPF during an ImageReceived event.

In C# Image.Source is set to a source generated by BitmapBuffer, which is a hack-way of accessing the raw data of the bitmap source: See this link. I do a P/Invoke of CopyMemory to copy the data from the Bitmap.Scan0 into the BitmapBuffer. I then invalidate the Image to update the screen, and Dispose() the Drawing.Bitmap object to free the memory.

While this method has worked for a while, it seems very hacky and I find it hard to believe that there is no other "appropriate" way to do this than through reflection.

Question: Is there a better way?

Spirited

Answer 1

Here's some code I wrote* for aliasing (sharing memory) between a WPF BitmapSource and a GDI Bitmap (for my own project)

Obviously, you'll need to tweak it for your own needs, it'll probably end up with a less "hacky" feel in the end.

class AliasedBitmapSource : BitmapSource {
    private Bitmap source;
    public AliasedBitmapSource(Bitmap source) {
        this.source = source;
        this.pixelHeight = source.Height;
        this.pixelWidth = source.Width;
        this.dpiX = source.HorizontalResolution;
        this.dpiY = source.VerticalResolution;
    }

    public override event EventHandler DownloadCompleted;
    public override event EventHandler<ExceptionEventArgs> DownloadFailed;
    public override event EventHandler<ExceptionEventArgs> DecodeFailed;

    protected override Freezable CreateInstanceCore() {
        throw new NotImplementedException();
    }

    private readonly double dpiX;
    public override double DpiX {
        get {
            return dpiX;
        }
    }

    private readonly double dpiY;
    public override double DpiY {
        get {
            return dpiY;
        }
    }

    private readonly int pixelHeight;
    public override int PixelHeight {
        get {
            return pixelHeight;
        }
    }

    private readonly int pixelWidth;
    public override int PixelWidth {
        get {
            return pixelWidth;
        }
    }

    public override System.Windows.Media.PixelFormat Format {
        get {
            return PixelFormats.Bgra32;
        }
    }

    public override BitmapPalette Palette {
        get {
            return null;
        }
    }

    public unsafe override void CopyPixels(Int32Rect sourceRect, Array pixels, int stride, int offset) {
        BitmapData sourceData = source.LockBits(
        sourceRect.ToRectangle(),
        ImageLockMode.ReadWrite,
        System.Drawing.Imaging.PixelFormat.Format32bppArgb);

        fixed (byte* _ptr = &((byte[])pixels)[0]) {
            byte* dstptr = _ptr;
            byte* srcptr = (byte*)sourceData.Scan0;

            for (int i = 0; i < pixels.Length; ++i) {
                *dstptr = *srcptr;
                ++dstptr;
                ++srcptr;
            }
        }

        source.UnlockBits(sourceData);
    }
}

public static class Extensions {
    public static Rectangle ToRectangle(this Int32Rect me) {
        return new Rectangle(
        me.X,
        me.Y,
        me.Width,
        me.Height);
    }

    public static Int32Rect ToInt32Rect(this Rectangle me) {
        return new Int32Rect(
        me.X,
        me.Y,
        me.Width,
        me.Height);
    }
}

*by "wrote" I mean "threw together in 10 minutes"

Nitride

Using Marshal.Copy instead unsafe code is even faster and does not requiere unsafe blocks. – Hoitytoity

Answer 2

If you are using the lastest WPF bits check out WriteableBitmap, you'll have to do more of the leg work but you'll really fast updates.

Do a quick google and you'll get some samples.

Unexacting

You'll end up using more memory and CPU cycles. Since the desired source is a GDI bitmap, it'll have to be copied to the WriteableBitmap every time it's updated. The AliasedBitmap approach above will automagically reflect changes made to either bitmap in the other. – Nitride

The only additional memory you'd consume is for the back buffer; but the advantage is that you'd avoid tearing. As far as CPU cycles, the memory copy seems to be about the same to me, you have to get the bitmap data from your app to the display - and the CopyPixels logic is almost the same in either. – Unexacting

Taking a look back at this, I believe years later, I ended up moving more towards this solution. I kept a single WriteableBitmap, doing a Buffer.BlockCopy to the BackBuffer from the ultrasound machine's image buffer. As a result, there was no memory usage over time (nothing being allocated) and even with a slower dual core processor, there was reliable performance around 60FPS. – Spirited

IMPORTANT: If you use the sample in the above MSDN link, make sure to change the line "int pBackBuffer = (int)writeableBitmap.BackBuffer;" to use (long). Otherwise, it will crash in 64-bit systems. – User