OCR with perceptron neural network of Aforge.net answers wrong

Question

I tried to make OCR by perceptrons with Aforge.Net in C#. I learned my network with nine 30*30 pictures in binary. But in the results, it recognizes everything as 'C'. this is the code:

    private void button1_Click(object sender, EventArgs e)
    {
        AForge.Neuro.ActivationNetwork network = new AForge.Neuro.ActivationNetwork(new AForge.Neuro.BipolarSigmoidFunction(2), 900, 3);
        network.Randomize();
        AForge.Neuro.Learning.PerceptronLearning learning = new AForge.Neuro.Learning.PerceptronLearning(network);
        learning.LearningRate =1 ;
        double[][] input = new double[9][];
        for (int i = 0; i < 9; i++)
        {
            input[i] = new double[900];
        }
   //Reading A images
        for (int i = 1; i <= 3; i++)
        {
            Bitmap a = AForge.Imaging.Image.FromFile(path + "\\a" + i + ".bmp");
            for (int j = 0; j < 30; j++)
                for (int k = 0; k < 30; k++)
                {
                    if (a.GetPixel(j, k).ToKnownColor() == KnownColor.White)
                    {
                        input[i-1][j * 10 + k] = -1;
                    }
                    else
                        input[i-1][j * 10 + k] = 1;
                }
           // showImage(a);

        }
   //Reading B images
        for (int i = 1; i <= 3; i++)
        {
            Bitmap a = AForge.Imaging.Image.FromFile(path + "\\b" + i + ".bmp");
            for (int j = 0; j < 30; j++)
                for (int k = 0; k < 30; k++)
                {
                    if (a.GetPixel(j , k).ToKnownColor() == KnownColor.White)
                    {
                        input[i + 2][j * 10 + k] = -1;
                    }
                    else
                        input[i + 2][j * 10 + k] = 1;
                }
           // showImage(a);

        }
   //Reading C images
        for (int i = 1; i <= 3; i++)
        {
            Bitmap a = AForge.Imaging.Image.FromFile(path + "\\c" + i + ".bmp");
            for (int j = 0; j < 30; j++)
                for (int k = 0; k < 30; k++)
                {
                    if (a.GetPixel(j , k ).ToKnownColor() == KnownColor.White)
                    {
                        input[i + 5][j * 10 + k] = -1;
                    }
                    else
                        input[i + 5][j * 10 + k] = 1;
                }
           // showImage(a);

        }

        bool needToStop = false;
        int iteration = 0;
        while (!needToStop)
        {
            double error = learning.RunEpoch(input, new double[9][] { new double[3] { 1, -1, -1 },new double[3] { 1, -1, -1 },new double[3] { 1, -1, -1 },//A
                new double[3] { -1, 1, -1 },new double[3] { -1, 1, -1 },new double[3] { -1, 1, -1 },//B
                new double[3] { -1, -1, 1 },new double[3] { -1, -1, 1 },new double[3] { -1, -1, 1 } }//C
                    /*new double[9][]{ input[0],input[0],input[0],input[1],input[1],input[1],input[2],input[2],input[2]}*/
                );
            //learning.LearningRate -= learning.LearningRate / 1000;
            if (error == 0)
                break;
            else if (iteration < 1000)
                iteration++;
            else
                needToStop = true;
            System.Diagnostics.Debug.WriteLine("{0} {1}", error, iteration);
        }
        Bitmap b = AForge.Imaging.Image.FromFile(path + "\\b1.bmp");
    //Reading A Sample to test Netwok
        double[] sample = new double[900];
        for (int j = 0; j < 30; j++)
            for (int k = 0; k < 30; k++)
            {
                if (b.GetPixel(j , k ).ToKnownColor() == KnownColor.White)
                {
                    sample[j * 30 + k] = -1;
                }
                else
                    sample[j * 30 + k] = 1;
            }
        foreach (double d in network.Compute(sample))
            System.Diagnostics.Debug.WriteLine(d);//Output is Always C = {-1,-1,1}
    }

I really wanted to know why it answers wrong.

Kragh

Hi, Did you find a solution for this problem? – Glottic

Answer 1

While loading your initial 30x30 images into a double[900] array in the input structure you are using the following calculation:

for (int j = 0; j < 30; j++)
    for (int k = 0; k < 30; k++)
    {
        if (a.GetPixel(j, k).ToKnownColor() == KnownColor.White)
            input[i-1][j * 10 + k] = -1;
        else
            input[i-1][j * 10 + k] = 1;
    }

Your offset calculation is wrong here. You need to change j * 10 + k to j * 30 + k or you will get invalid results. Later you use the correct offset calculation while loading the test image, which is why it's not being matched correctly against the corrupted samples.

You should write a method to load a bitmap into a double[900] array and call it for each image, instead of writing the same code multiple times. This helps to reduce problems like this, where different results are given by two pieces of code that should return the same result.

Benzocaine

Answer 2

I tried your code. It helped me too and thanks a lot for that. I could get your code working by doing some changes to getting bit array from the image. Here's the method I used.

`
        private double[] GetImageData(Bitmap bmp)
        {
        double[] imageData = null;

        //Make the image grayscale
        Grayscale filter = new Grayscale(0.2125, 0.7154, 0.0721);
        bmp = filter.Apply(bmp);

        //Binarize the image
        AForge.Imaging.Filters.Threshold thFilter = new AForge.Imaging.Filters.Threshold(128);
        thFilter.ApplyInPlace(bmp);

        int height = bmp.Height;
        int width = bmp.Width;
        imageData = new double[height * width];
        int imagePointer = 0;
        System.Diagnostics.Debug.WriteLine("Height : " + height);
        System.Diagnostics.Debug.WriteLine("Width  : " + width);

        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                System.Diagnostics.Debug.Write(string.Format("({0}  , {1})     Color : {2}\n", i, j, bmp.GetPixel(i, j)));

                //Identify the black points of the image
                if (bmp.GetPixel(i, j) == Color.FromArgb(255, 0,  0, 0))
                {
                    imageData[imagePointer] = 1;
                }
                else
                {
                    imageData[imagePointer] = 0;
                }
                imagePointer++;
            }
            System.Diagnostics.Debug.WriteLine("");
        }
        System.Diagnostics.Debug.WriteLine("Bits  : " + imagePointer );
        return imageData;
    }`

Hope this will help. Thanks.

Glottic

Answer 3

try this

double error = learning.RunEpoch(input, new double[9][] { new double[3] **{ 1, -1, -1 }**,new double[3] **{ -1, 1, -1 }**,new double[3] **{ -1, -1, 1 }**,//A
                new double[3] **{ 1, -1, -1 }**,new double[3] **{ -1, 1, -1 }**,new double[3] **{ -1, -1, 1 }**,//B
                new double[3] **{ 1, -1, -1 }**,new double[3] **{ -1, 1, -1 }**,new double[3] **{ -1, -1, 1 }** }//C

                );

or this way

double[][] output = new double[patterns][];
            for (int j = 0; j < patterns; j++)
            {
                output[j] = new double[patterns];
                for (int i = 0; i < patterns; i++)
                {
                    if (i != j)
                    {
                        output[j][i] = -1;
                    }
                    else
                    {
                        output[j][i] = 1;
                    }
                }
            }


double error = learning.RunEpoch(input,output)

double[] netout = neuralNet.Compute(pattern);

 int maxIndex = 0;
            double max = netout[0];

            for (int i = 1; i < netout.Length; i++)
            {
                if (netout[i] > max)
                {
                    max = netout[i];
                    maxIndex = i;
                }
            }

if maxIndex=0 answer is A

if maxIndex=1 answer is B

if maxIndex=2 answer is C

also I think that you must create matrix from images and use it as pattern, for example 20/20 or 15/15 or small, your 30/30 is big.

I use different way for get Image Scheme. I Divide image 20/20 and If one of pixels in rectangle is black (or another colour you want) save 1 in matrix, else 0.

I make replace all pixels an after this I have only two colours, White and Black, I can manipulate with contour.

private void Cmd_ReplaceColors(ref WriteableBitmap Par_WriteableBitmap,int Par_Limit=180)
        {

            for (int y = 0; y < Par_WriteableBitmap.PixelHeight; y++)
            {
                for (int x = 0; x < Par_WriteableBitmap.PixelWidth; x++)
                {

                    Color color = Par_WriteableBitmap.GetPixel(x, y);

                    if (color == Colors.White)
                    {

                    }
                    else
                    {
                        if (color.R < Par_Limit)
                        {
                            Par_WriteableBitmap.SetPixel(x, y, Colors.Black);
                        }
                        else
                        {
                            Par_WriteableBitmap.SetPixel(x, y, Colors.White);
                        }

                    }

                }
            }

            Par_WriteableBitmap.Invalidate();
        }

1000 iterations in my opinion is small, better 100 000 :)

Orange