I was trying out a weird 2-hidden layer Autoencoder, where instead of 1 hidden layer on an typical AutoEncoder, I have 2. Note, this is not a stacking up of 2 autoencoders, but rather training the 2 hidden layer parameters at one go. I wanted to try this just to compare the results with typical autoencoders, no other reason.
I found that my filters (between the visible and the 1st hidden layer) were not "sharp". As with Zeiler's paper on deconvolutional networks for visualizing the Krizhevsky-net, sharp filters are often a good sign for the network.
So I want to diagnose what the reason behind blurry filters could be. Would anyone have an idea of typical reasons for this? like a lack of data, training epochs, or a learning rule (Adagrad/Adadelta..)? I'll be trying out these but advice from someone who already has explored this would be very helpful.
[link][2 comments]