I've been trying to get an RBM up because I want to be sure I'm understanding it well enough before using a library. I have one now which seems to be learning, but isn't recreating linear values on the visible side. I think my learning function is incorrect, but the error is decreasing across epochs. I've tried averaging the runs rather than applying the weights immediately, but it only seems to smooth out the learning.
The reason I suspect it's not learning right is as follows. If I train it on a linear value, say (x, 2*x) for all x in the (0, 1000) or (0,1) with step 0.01, and push some value like (1.0, 1.0), I don't get back (1.0, 2.0). I get either 1,1 if I'm activating on the real like, or I get something like 2.0e-10, 1.4e-10 if I use real valued numbers.
I'd appreciate any hints about what I'm doing wrong, or advice on how to test the validity of the RBM.
Source: http://pastebin.com/KytDMwUz
[link] [2 comments]