In order to learn about deep learning and NNs I've been trying to roll my own NN code (in addition to going through some coursera videos and reading papers and even a textbook). I've found that the devil really is in the details and I've often stumbled on problems whose answer I can't seem to find in the above mentioned resources.
For instance:
I've been trying to train a deep autoencoder in a layer-wise fashion (for a Kaggle competition) but I don't have any sense for what a reasonable (layer-wise) training error on an auto-encoder is. Obviously it can't be expected to reproduce the input data fully faithfully but I'd like some heuristic measure of what a good architecture is.
To speed things up I tried to use gnumpy (as a drop-in for numpy) but despite various improvements I find the problem set is either too small for the GPU to beat the CPU or its too big to fit in memory. I know this is wrong and I'm sure other people have had similar issues and it would be useful to get their insights.
I should add (just to be clear) that I'm not out to write another ML library -- I'm implementing stuff myself to learn it.
Anyway, I would like to be discuss my problems with experienced people or other people in the same position but I'm not sure what the right forum is. These questions seem a bit too detailed and technical for this subreddit so I would like to ask for any other recommendations on where to go for help (or confirmations that this is a good place to ask).
[link][2 comments]