What is the relationship between bayesian and neural networks?

Question

I'm looking for computationally heavy tasks to implement with CUDA and wonder if neural networks or bayesian networks might apply. This is not my question, though, but rather what the relation between the two network types is. They seem very related, especially if you look at bayesian networks with a learning capability (which the article on wikipedia mentions). At a glance, bayesian networks look at bit like a specific type of neural networks. Can anyone sum up their relationship, and if there is any connection beyond the apparent similarity?

Situated

should migrate to stats.stackexchange.com/questions – Hairdresser

stats.stackexchange.com/questions/94511/… – Hairdresser

Answer 1

Bayesian networks represent independence (and dependence) relationships between variables. Thus, the links represent conditional relationships in the probabilistic sense. Neural networks, generally speaking, have no such direct interpretation, and in fact the intermediate nodes of most neural networks are discovered features, instead of having any predicate associated with them in their own right.

Idzik

Hmmm ... I always had the impression that bayesian statistics was all about discovering not assigning relationships. – Reservoir

That's right, to an extent, and thus there's a difference between learning the structure of a Bayesian network, and learning a given network's parameters. It turns out that you can learn several plausible structures, but it's hard to know if you are missing variables. See "Causality" by Judea Pearl – Idzik

Actually, the more I think about it, the key point to emphasize is that the nodes and links are connected through Bayes rule, and as such are subject to Bayesian updates upon further data. Beyond this, I'm not sure to what can appropriately be ascribed as Bayesian. – Idzik

Answer 2

It is reported that Bayesian networks are more resistant to the "overtraining" that is seen in some neural networks. In other words some neural networks become so "trained" to the observed measurements used in training that they aren't useful for the general cases.

Intisar

Answer 3

Indeed they are. I see a bayesian network as a neural network applying the Baye's Theorem on large scale, but I don't remember details. I know where you can find them and I recommend this book for that.

Julietajulietta

This is not true. – Maremma