Find an image in an image C#

Question

I have a scan of a document (a form actually) filled with some handwritten infos.

I have a bitmap of the form empty.

how can i "cancel" the printed form to extract the handwritting only.

I use C#... Thanks Jonathan

Shuffle

Answer 1

What you want to do is subtract the empty form image from the image of the form with handwriting in it. This will give you a reasonable image of the handwriting alone.

Please note that this will not register the images. Registration will line them up so that they are at identical orientations to give the subtraction the best chance of success. If your images are poorly aligned you'll have to look into image registration.

Here's a snippet of code I wrote a while back to do something similar (this code highlights differences in red):

        Bitmap b1 = new Bitmap(fname1);
        Bitmap b2 = new Bitmap(fname2);

        if (b1.Height != b2.Height || b1.Width != b2.Width) {
           MessageBox.Show("Input files are not the same dimensions!");
           Application.Exit();
        }

        totalPixels = b1.Height * b1.Width * 4;

        Bitmap outImg = new Bitmap(b1.Width, b1.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb);

        BitmapData b1Data = b1.LockBits(new Rectangle(0, 0, b1.Width, b1.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
        BitmapData b2Data = b2.LockBits(new Rectangle(0, 0, b1.Width, b1.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
        BitmapData oData = outImg.LockBits(new Rectangle(0, 0, b1.Width, b1.Height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppRgb);

        byte[] cur1 = new byte[b1Data.Stride * b1Data.Height];
        byte[] cur2 = new byte[b2Data.Stride * b2Data.Height];
        byte[] curOut = new byte[b2Data.Stride * b2Data.Height];

        Marshal.Copy(b1Data.Scan0, cur1, 0, b1Data.Stride * b1Data.Height);
        Marshal.Copy(b2Data.Scan0, cur2, 0, b2Data.Stride * b2Data.Height);

        for (int i = 0; i < b1Data.Stride * b1Data.Height; i += 4) {
           byte temp1 = cur1[i], temp2 = cur2[i], first = 0, second = 0;
           curOut[i] = 0;
           first = (byte) ((temp1 > temp2) ? temp1 - temp2 : temp2 - temp1);

           temp1 = cur1[i + 1];
           temp2 = cur2[i + 1];
           curOut[i + 1] = 0;
           second = (byte) ((temp1 > temp2) ? temp1 - temp2 : temp2 - temp1);

           temp1 = cur1[i + 2];
           temp2 = cur2[i + 2];
           curOut[i + 2] = (byte) ((temp1 > temp2) ? temp1 - temp2 : temp2 - temp1);
           curOut[i + 2] = (byte) ((first + second + curOut[i + 2]) * 255);

           curPixel = i;
        }

        Marshal.Copy(curOut, 0, oData.Scan0, b2Data.Stride * b2Data.Height);

        b1.UnlockBits(b1Data);
        b2.UnlockBits(b2Data);
        outImg.UnlockBits(oData);

        outImg.Save(outfile);

Backflow

Yes, see my edit regarding registration. If you have images that are very close registration may be as simple as applying a small rotation or modifying the histogram. If they are farther apart, you'll need to look into projecting them to a uniform space for comparison. – Backflow

Sid. What is image registration ? Can you point me to some info on that. Did you try also to apply recognition on the text?Thanks ! – Shuffle

Handwriting recognition is a very different problem than simply extracting the information from the image. I suggest you look into a 3rd party solution to do that for you as there are commercial solutions that still don't do a great job. As for image registration the wikipedia article is a great start en.wikipedia.org/wiki/Image_registration – Backflow

Oh yes, i have third party tools for that. thanks for the ideas, the code and the link. – Shuffle

If data entered into the fields overlaps the fields themselves (i.e. handwriting outside of margins), subtracting out the empty forms will also erase the overlapping regions. – Howrah

Answer 2

As an alternative (and possibly much faster method) could you not just store the rectangle psoitions of where the "fields" are going to be, then simply extract the pixels for each rectangles?

Darknight

Ceria

this may be a good solution but people seldom write at the right position – Shuffle

Might work if the scanned documents are always aligned the same way. Points for simplicity. +1 – Ardellaardelle

Also a good method, it may be easier to get rectangle regions aligned to the field areas than to align the entire form. Simple heuristics could eliminate fragments on the edge of the rectangle and possibly expand it if the text was deemed 'cut off'. – Backflow