Computer Science · cs.NE
Learning Activation Functions to Improve Deep Neural Networks
Forest Agostinelli, Matthew Hoffman, Peter Sadowski, Pierre Baldi
- The canon
- #80 in Neural and Evolutionary Computing (cs.NE)
- #11 of 2014 in cs.NE
- top 4% of Computer Science
Abstract
Artificial neural networks typically have a fixed, non-linear activation function at each neuron. We have designed a novel form of piecewise linear activation function that is learned independently for each neuron using gradient descent. With this adaptive activation function, we are able to improve upon deep neural network architectures composed of static rectified linear units, achieving state-of-the-art performance on CIFAR-10 (7.51%), CIFAR-100 (30.83%), and a benchmark from high-energy physics involving Higgs boson decay modes.
References
The reference list for this paper has not been collected yet.
Cited by 3
3 papers in the index cite this (Semantic Scholar counts 521 in total).
The cited-by list is still filling in while reference lists are collected.