Hi!
I am trying to teach a multilayer perceptron to do an auto-association task but I can't get it to work properly. I have little experience with neural networks so I thought that the solution to my problem could be obvious for you guys. I tried many sets of parameters (size of hidden layer and learning rate) on my relatively simple implementation of the backprop rule, but without success. My inputs are images of black and white fish. The problem that I have is that the cost always seems to plateau after a while, and instead of giving me the same output as my input after the training, I get a kind of prototype fish for all inputs. I tried to lower my learning rate but it doesn't seem to be helping.
I have a sigmoid activation for my hidden and output units. My inputs are all vector with 8064 binary features. My cost function is the following:
def cost(pred,y,_lambda,w1,w2): m = pred.shape[0] J = np.sum(np.power(pred - y,2))/(2*m) if _lambda != 0: regularization = (_lambda/(2*m)) * (np.sum(np.power(w1[:,1:],2)) + np.sum(np.power(w2[:,1:],2))) J = J + regularization return J Thank you!
[link][comment]