Passing bitmap from c# to c++

Question

I have an image processing function written in C++ based on opencv. In my wpf application I have used AForge library to access a webcam and update it on UI. This the function for handling newframes.

void UI_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        try
        {
            System.Drawing.Bitmap bitmapFrame = (Bitmap)eventArgs.Frame.Clone();

            MemoryStream ms = new MemoryStream();
            bitmapFrame.Save(ms, ImageFormat.Bmp);
            ms.Seek(0, SeekOrigin.Begin);

            BitmapImage bitmapImageFrame = new BitmapImage();
            bitmapImageFrame.BeginInit();
            bitmapImageFrame.StreamSource = ms;
            bitmapImageFrame.EndInit();
            bitmapImageFrame.Freeze();
            CurrentFrame = bitmapImageFrame;
        }
        catch (Exception ex)
        {
            Debug.WriteLine("fatal::" + ex.Message);
        }
    }

This my c++ code:

void FaceTracker(unsigned char* imageBuffer, int width, int height){    
   Mat frame(Size(width, height), CV_8UC4, imageBuffer, Mat::AUTO_STEP);
   Mat gray;
   cvtColor(frame, gray, COLOR_BGR2GRAY);
   //do more operations
}

I have two options. I have already found a code for copying data from BitmapImage to a IntPtr buffer using CopyPixels. I tested it and it works fine. But I also read I can simply pass a handle to the Bitmap using GetHBitmap. However this is not working for me. I read this How can I pass a .NET Bitmap to a native DLL? but I don't understand the part about GetDIBits. I also tried locking pixels of bitmap and passing like this:

bitmapFrame.LockBits(new Rectangle(0, 0, bitmapFrame.Width, bitmapFrame.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, 
            System.Drawing.Imaging.PixelFormat.Format24bppRgb);
NativeFunctions.FaceTracker(bitmapFrame.GetHbitmap(), bitmapFrame.Width, bitmapFrame.Height);

and simply passing HBitMap, neither works. I always get an exception stating memory access violation.

Chambray

Just use LockBits, and pass Scan0. GetHbitmap is not at all the same as Scan0. – Handbook

Answer 1

Depends on the scope. If you can guarantee the bitmap isn't used elsewhere, you can lock the image buffer, and then pass the pointer down to the C++ code, and unlock it afterwards.

The LockBits command returns a BitmapData class that has a pointer to the image buffer in its Scan0 property:

BitmapData bmpData = bitmapFrame.LockBits(new Rectangle(0, 0, bitmapFrame.Width, bitmapFrame.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, 
            System.Drawing.Imaging.PixelFormat.Format24bppRgb);

NativeFunctions.FaceTracker(bmpData.Scan0 , bitmapFrame.Width, bitmapFrame.Height);

bitmapFrame.UnlockBits(bmpData); //Remember to unlock!!!

Lath

Exactly what I was looking for. As I had hoped, passing Bitmap instead of BitmapImage gave me a notable decrease in cpu use. Thank you :) – Chambray

Dont forget to pass the bmpData.Stride as well. if you dont specify the step size in the C++ part, you will get distorted images (when dealing with color images) like this question : #45739218 Also you should be using bitmapFrame..PixelFormat instead of hard-coding it . – Fetlock

I was looking for days until I found this answer others need to upvote this one !!! It didn't give me the final solution but the data transfer is spot on :) – Gravy

Answer 2

An HBITMAP is an opaque handle to a bitmap. It is not a pixel buffer. So your two pieces of code do not match.

If you pass an HBITMAP to the native code then you need the native code to use GDI functions to obtain the pixel buffer and operate on it. Alternatively you could obtain a pixel buffer in your managed code and pass that to the native code. Whichever way you go you do need to match the two sides of the interop.

Makepeace

I am assuming GetDIBits, CreateCompatibeDC are GDI functions? I would prefer to convert my bitmap to pixel buffer and pass to native code. How would I do that? – Chambray

LockBits and then Scan0 – Makepeace

Answer 3

You can use Bitmap.Save() method to convert it to a memory stream and then send it to C++ dll.

public Image ConvertImage(Image image)
{
   MemoryStream convertedImageMemoryStream;
   using (MemoryStream sourceImageStream = new MemoryStream())
   {
       image.Save(sourceImageStream, System.Drawing.Imaging.ImageFormat.Png);
       byte[] sourceImageData = sourceImageStream.ToArray();

       // Send it to dll and get the IntPtr byte array from dll

       byte[] imageData = new byte[imInfo.size];
       Marshal.Copy(imInfo.data, imageData, 0, imInfo.size);
       if (imInfo.data != IntPtr.Zero)
           AlgorithmCpp.ReleaseMemoryFromC(imInfo.data);
       convertedImageMemoryStream = new MemoryStream(imageData);
    }
    Image processed = new Bitmap(convertedImageMemoryStream);
    return processed;
}

Then, in C++ dll use decoding such as cv::imdecode() method to get the image.

DllExport void convertToGray(unsigned char* data, int dataLen)
{
    vector<unsigned char> inputImageBytes(data, data + dataLen);
    Mat image = imdecode(inputImageBytes, CV_LOAD_IMAGE_COLOR);
    Mat processed;
    cvtColor(image, processed, CV_BGR2GRAY);
    vector<unsigned char> bytes;
    imencode(".png", processed, bytes);
    // ....

There are some pros in this way such as, 1. fewer data to transfer 2. minimum implementation effort from both C# and C++ end. 3. Does not depend on the source image type eg. color or grayscale or any other color palate type. So it's very safe to use.

The only problem is that it is CPU intensive as there are an encoding and a decoding.

Details are here.

Merkle

Happy that it helps. :) – Merkle

Answer 4

I can suggest you to use EmguCV. It is wrapper for openCV in C#. You dont have to worry about dll, lib etc. I can see that you want to track a face from webcamera. EmguCV gives you possibilty to capture webcamera image and later track face in very easy way: http://www.emgu.com/wiki/index.php/Face_detection This link should help you also. Enjoy.

Disrupt

Yes I read about EmguCV. But the reason I am writing my own code is because I am a fresh out of college developer and my position requires me to work with interop a lot in the future. So I have to learn all its intricacies :) – Chambray