Maxout uses the max-norm constraint (or dropout does). Do you know when to enforce the constraint, during update ? or during calculating the pre-activation?
[link][3 comments]
Maxout uses the max-norm constraint (or dropout does). Do you know when to enforce the constraint, during update ? or during calculating the pre-activation?