I'm interested in playing around with alternatives to dropout for a course project. However, before I do that it'd be nice to know if anyone has gotten dropout working on a smaller/medium sized dataset with a convolutional network small enough to run in MATLAB/python. Have you gotten dropout to work? If so, what dataset, network, and optimization algorithm did you use?
Just for discussion, anyone have an opinion on dropout? Seems to be getting a lot of attention within deep learning. I believe we'll need better regularization techniques for these larger networks, especially on small datasets, but it's not clear to me at what scales dropout becomes useful.
[link][4 comments]