Quantcast
Viewing all articles
Browse latest Browse all 62908

How does regularization affect dropout while training autoencoders?

I was reading this paper on dropout:

http://arxiv.org/pdf/1207.0580v1.pdf

...And after a rather straightforward explanation of what dropout is and what it is supposed to do, they use the third paragraph to detail a threshold-based regularization procedure that they use, instead of the typical L2 norm weight penalty. Unless I missed something in the paper, the authors never really come back to this or explain if it has any fundamental role in the effectiveness of dropout.

I'm open to having missed something, but is a threshold-based regularization procedure weight penalty a wholly independent aspect of training, or does it have some interaction with Dropout? In either case, they also don't seem to mention what an appropriate value for the weight divisor is after the L2 breaches said threshold, or the value of the threshold itself.

Edit: Regularization is the wrong word for penalizing weight growth. I'm aware that dropout is itself a regularization procedure. Post was written pre-coffee.

submitted by eubarch
[link][2 comments]

Viewing all articles
Browse latest Browse all 62908

Trending Articles