Encog C# RBF network, how to start?

Question

I went through whole documantation and didnt find how to set RBF network. I found some RBF example in ConsoleExmpales/Examples/Radial, but it looks like it doesnt work anymore, beceause some methods have been changed in Encog.

So far I am stuck on this:

    public static double[][] XORInput = {
        new[] {0.0, 0.0},
        new[] {1.0, 0.0},
        new[] {0.0, 1.0},
        new[] {1.0, 1.0}
    };

    public static double[][] XORIdeal = {
        new[] {0.0},
        new[] {1.0},
        new[] {1.0},
        new[] {0.0}
    };

        int dimension = 8;
        int numNeuronsPerDimension = 64;
        double volumeNeuronWidth = 2.0 / numNeuronsPerDimension;
        bool includeEdgeRBFs = true;

        RBFNetwork n = new RBFNetwork(dimension, numNeuronsPerDimension, 1, RBFEnum.Gaussian);
        n.SetRBFCentersAndWidthsEqualSpacing(0, 1, RBFEnum.Gaussian, volumeNeuronWidth, includeEdgeRBFs);
        //n.RandomizeRBFCentersAndWidths(0, 1, RBFEnum.Gaussian);

        INeuralDataSet trainingSet = new BasicNeuralDataSet(XORInput, XORIdeal);
        SVDTraining train = new SVDTraining(n, trainingSet);

        int epoch = 1;
        do
        {
            train.Iteration();
            Console.WriteLine("Epoch #" + epoch + " Error:" + train.Error);
            epoch++;
        } while ((epoch < 1) && (train.Error > 0.001));

When I run this, I get error "Total number of RBF neurons must be some integer to the power of 'dimensions'." on SetRBFCentersAndWidthsEqualSpacing. It works if I change this method for RandomizeRBFCentersAndWidths until train.iteration() is reached, where i get " Index was outside the bounds of the array".

I understand how RBF network works, but I am confused from all parameters in SetRBFCentersAndWidthsEqualSpacing method, can someone explain it more detail?.

Cymene

Answer 1

Very good question.

1. SetRBFCentersAndWidthsEqualSpacing and here is relatively new method of training neural networks and Jeff Heaton decided to implement it.

2. Looks like there is a difference between Java version and C# version at lines 230 - 240 and IMHO bug sits in Java version.

3. I've modified your code in order for it to be workable with additional comments:

   using System;
   using System.Collections.Generic;
   using System.Linq;
   using System.Text;
   using System.Threading.Tasks;
   using Encog.MathUtil.RBF;
   using Encog.Neural.Data.Basic;
   using Encog.Neural.NeuralData;
   using Encog.Neural.Rbf.Training;
   using Encog.Neural.RBF;
   
   namespace TestRBF
   {
       class Program
       {
           public static double[][] XORInput = {
           new[] {0.0, 0.0},
           new[] {1.0, 0.0},
           new[] {0.0, 1.0},
           new[] {1.0, 1.0}
       };
   
           public static double[][] XORIdeal = {
           new[] {0.0},
           new[] {1.0},
           new[] {1.0},
           new[] {0.0}
       };
   
           static void Main(string[] args)
           {
               int dimension = 2; // XORInput provides two-dimensional inputs. Not 8. 
               /*
               If XORInput is  8 dimensional  it should be like this:
   
               public static double[][] XORInput = {
               new[] {0.0, 0.0,0.0, 0.0,0.0, 0.0,0.0, 0.0}, 
               .
               .   
               .*/
               int numNeuronsPerDimension = 4; // could be also 16, 64, 256. I suppose it should accept 8, 32 but it needs additional investigation
               double volumeNeuronWidth = 2.0 / numNeuronsPerDimension;
               bool includeEdgeRBFs = true;
   
               RBFNetwork n = new RBFNetwork(dimension, numNeuronsPerDimension, 1, RBFEnum.Gaussian);
               n.SetRBFCentersAndWidthsEqualSpacing(0, 1, RBFEnum.Gaussian, volumeNeuronWidth, includeEdgeRBFs);
               //n.RandomizeRBFCentersAndWidths(0, 1, RBFEnum.Gaussian);
   
               INeuralDataSet trainingSet = new BasicNeuralDataSet(XORInput, XORIdeal);
               SVDTraining train = new SVDTraining(n, trainingSet);
   
               int epoch = 1;
               do
               {
                   train.Iteration();
                   Console.WriteLine("Epoch #" + epoch + " Error:" + train.Error);
                   epoch++;
               } while ((epoch < 1) && (train.Error > 0.001));
   
           }
       }
   }
   

Shortage

I forgot to change my dimension, 2 is right. Thank you for point to numNeuronsPerDimension, now it works! I have tried 8, 32 and it doesnt work, so only 16, 64, 256. – Cymene

WRT power of dimension is requirement of RBF centers and equal widths spacing algorithm – Shortage

I think there is a bug in .NET version on the line 232: var expectedSideLength = (int) Math.Pow(totalNumHiddenNeurons, 1.0d/dimensions); double cmp = Math.Pow(totalNumHiddenNeurons, 1.0d/dimensions); if (expectedSideLength != cmp) these two variables can't be equal, because "(int)" rounds the number. It's coincidence that it works for XOR example, with different dimenson like 19 it won't work. – Cymene

I propose to create another SO thread for that question mark. Hopefully Jeff Heaton will join us in that analysis. – Shortage

I've made new thread, but I am not sure if Jeff Heaton still works on this project. #40073175 – Cymene