Artificial Neural Network that creates it's own connections

Question

I've been reading about feed forward Artificial Neural Networks (ANN), and normally they need training to modify their weights in order to achieve the desired output. They will also always produce the same output when receiving the same input once tuned (biological networks don't necessarily).

Then I started reading about evolving neural networks. However, the evolution usually involves recombining two parents genomes into a new genome, there is no "learning" but really recombining and verifying through a fitness test.

I was thinking, the human brain manages it's own connections. It creates connections, strengthens some, and weakens others.

Is there a neural network topology that allows for this? Where the neural network, once having a bad reaction, either adjusts it's weights accordingly, and possibly creates random new connections (I'm not sure how the brain creates new connections, but even if I didn't, a random mutation chance of creating a new connection could alleviate this). A good reaction would strengthen those connections.

I believe this type of topology is known as a Turing Type B Neural Network, but I haven't seen any coded examples or papers on it.

Constantia

Answer 1

This paper, An Adaptive Spiking Neural Network with Hebbian Learning, specifically addresses the creation of new neurons and synapses. From the introduction:

Traditional rate-based neural networks and the newer spiking neural networks have been shown to be very effective for some tasks, but they have problems with long term learning and "catastrophic forgetting." Once a network is trained to perform some task, it is difficult to adapt it to new applications. To do this properly, one can mimic processes that occur in the human brain: neurogenesis and synaptogenesis, or the birth and death of both neurons and synapses. To be effective, however, this must be accomplished while maintaining the current memories.

If you do some searching on google with the keywords 'neurogenesis artificial neural networks', or similar, you will find more articles. There is also this similar question at cogsci.stackexchange.com.

Creator

hey thanks, I forgot I even asked this. So much fun to see such a gem. – Constantia

Answer 2

neat networks as well as cascading add their own connections/neurons to solve problems by building structures to create specific responses to stimuli

Constantia

Although NEAT does manage its own connections, it does so through an evolutionary algorithm, which this question was specifically not asking about. – Hydromel