The below is how I organized my design and code when I was messing with neural networks. The code here is (obviously) psuedocode and roughly follows Object Oriented conventions.
Starting from the bottom up, you'll have your neuron. Each neuron needs to be able to hold the weights it puts on the incoming connections, a buffer to hold the incoming connection data, and a list of its outgoing edges. Each neuron needs to be able to do three things:
- A way to accept data from an incoming edge
- A method of processing the input data and weights to formulate the value this neuron will be sending out
- A way of sending out this neuron's value on the outgoing edges
Code-wise this translates to:
// Each neuron needs to keep track of this data
float in_data[]; // Values sent to this neuron
float weights[]; // The weights on each edge
float value; // The value this neuron will be sending out
Neuron out_edges[]; // Each Neuron that this neuron should send data to
// Each neuron should expose this functionality
void accept_data( float data ) {
in_data.append(data); // Add the data to the incoming data buffer
}
void process() {
value = /* result of combining weights and incoming data here */;
}
void send_value() {
foreach ( neuron in out_edges ) {
neuron.accept_data( value );
}
}
Next, I found it easiest if you make a Layer class which holds a list of neurons. (It's quite possible to skip over this class, and just have your NeuralNetwork hold a list of list of neurons. I found it to be easier organizationally and debugging-wise to have a Layer class.) Each layer should expose the ability to:
- Cause each neuron to 'fire'
- Return the raw array of neurons that this Layer wraps around. (This is useful when you need to do things like manually filling in input data in the first layer of a neural network.)
Code-wise this translates to:
//Each layer needs to keep track of this data.
Neuron[] neurons;
//Each layer should expose this functionality.
void fire() {
foreach ( neuron in neurons ) {
float value = neuron.process();
neuron.send_value( value );
}
}
Neuron[] get_neurons() {
return neurons;
}
Finally, you have a NeuralNetwork class that holds a list of layers, a way of setting up the first layer with initial data, a learning algorithm, and a way to run the whole neural network. In my implementation, I collected the final output data by adding a fourth layer consisting of a single fake neuron that simply buffered all of its incoming data and returned that.
// Each neural network needs to keep track of this data.
Layer[] layers;
// Each neural network should expose this functionality
void initialize( float[] input_data ) {
foreach ( neuron in layers[0].get_neurons() ) {
// do setup work here
}
}
void learn() {
foreach ( layer in layers ) {
foreach ( neuron in layer ) {
/* compare the neuron's computer value to the value it
* should have generated and adjust the weights accordingly
*/
}
}
}
void run() {
foreach (layer in layers) {
layer.fire();
}
}
I recommend starting with Backwards Propagation as your learning algorithm as it's supposedly the easiest to implement. When I was working on this, I had great difficulty trying to find a very simple explanation of the algorithm, but my notes list this site as being a good reference.
I hope that's enough to get you started!
