A characteristic feature of biological neural networks is that the connections between neurons are directed. Backpropagation as well as Boltzmann machines rely on bidirectional connections that allow information to flow in either direction unidirectional connections. In backpropagation the error signal is propagated backwards through the connections to the pre-synaptic neuron, while in Boltzmann machines the weights are assumed to be tied. However, I came across this slide in a talk by Geoffery Hinton, in which he claims that the weights in a Boltzmann machine do not necessarily need to be tied (see Reference). Can anyone refer me to a paper on this? Has someone derived the equations for a Boltzmann machine with purely directed connections?
Reference: http://www.cs.toronto.edu/~hinton/backpropincortex2007.pdf#page=12
Thanks!
[link][5 comments]