I've been trying to wrap my head around the Deep Learning scene for a while now, but I'm merely an enthusiast and get a little lost in the details once in a while.
Following the literature over the last few years, it seemed like deep methods like RBMs etc. had a way of leveraging unlabeled data for e.g. a classification task, which is otherwise a task involving labeled data. As far as I understood, pre-training made the network generatively model the data, and when coupled to your classification task (via backpropagating using the labels) this would outperform methods that learnt through the labels alone.
Then Hinton recently gave his formula for DREDNET, a deep network with rectified linear units and dropout, which seems to be a return to form for supervised neural networks.
My question, then, is: where does this leave semi-supervised models, particularly Bengio's Deep Generative Stochastic Networks? (Important enough to get a Wired article yet the only benchmark I can find is MNIST)
[link][1 comment]