arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Artificial Intelligence

EXPLORING ACTIVATION FUNCTIONS: A COMPREHENSIVE STUDY ON ENHANCING CONVENTIONAL NEURAL NETWORK LEARNING

S. Jayashree, B. Jansi, V. Sumalatha

See on the map

Abstract

Activation functions are essential components in neural networks as they introduce non-linearity, enabling the model to learn complex relationships in the data. Their role in enhancing the learning capabilities of conventional neural networks is crucial to achieve high performance in various tasks. This comprehensive study delves into the world of activation functions, examining their characteristics, advantages, and limitations, with a focus on enhancing the learning process of conventional neural networks. Various activation functions are meticulously analyzed to understand their impact on neural network performance. The traditional sigmoid and hyperbolic tangent (tanh) functions are explored, with discussions on their saturated regions and the vanishing gradient problem. Rectified Linear Units (ReLU) and its variants, such as Leaky ReLU and Parametric ReLU, are also studied for their ability to mitigate the vanishing gradient issue and accelerate convergence. It highlights the importance of selecting suitable activation functions and encourages the exploration of novel alternatives to further enhance the performance and robustness of neural networks in various domains.

More like this