Computer Science · Computer Vision and Pattern Recognition
An Analysis of Regularization Methods in Deep Neural Networks
Akshay Badola, Vineet Nair, Rajendra Prasad Lal
See on the mapAbstract
Regularization in Deep Neural Networks for Classification has developed into a separate paradigm as that involves regularization in probability spaces. A big contribution to avoid overfitting in Deep Learning has been Dropout [1]. Dropout however is rarely applied alone for classification tasks and is usually used in conjunction with several other techniques like weight normalization which is equivalent to l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm or batch normalization [2]. The use of these techniques is empirical in nature and often ad hoc and thus, it's difficult to estimate the contribution of each of these techniques to the final outcome. Here we isolate each of the common regularization techniques and use a standard Deep Convolutional Network VGG11 [3] and a standard dataset CIFAR10 [4]. We collect and analyze the results to identify the effect of each of these techniques.
Cited by 1
1 paper in the index cites this.
The cited-by list is still filling in while reference lists are collected.