arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Lassi Meronen

6 papers · 2020–2023

2020: 32021: 12022: 02023: 2
2020peak 3 in 20202023

Frequent co-authors

Papers

showing 1–6 of 6 RSS

Fixing Overconfidence in Dynamic Neural Networks

Lassi Meronen, Martin Trapp, Andrea Pilzer +2 more

cs.LG · · cited 28× · arXiv ↗ · more like this ·

Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the inputs. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100, ImageNet, and Caltech-256 in…

Stationary Activations for Uncertainty Calibration in Deep Learning

Lassi Meronen, Christabella Irwanto, Arno Solin

cs.LG · · cited 7× · arXiv ↗ · more like this ·

We introduce a new family of non-linear neural network activation functions that mimic the properties induced by the widely-used Matérn family of kernels in Gaussian process (GP) models. This class spans a range of locally stationary models of various degrees of mean-square differentiability. We show an explicit link to the corresponding GP models in the case that the network consists of one infinitely wide hidden layer. In the limit of infinite smoothness the Matérn family results in the RBF kernel, and in this case we recover RBF activations. Matérn activation functions result in similar appealing properties to their counterparts in GP models, and we demonstrate that the local stationarity property together with limited mean-square differentiability shows both good performance and uncertainty calibration in Bayesian deep learning tasks. In particular, local stationarity helps…

Epävarmuuden arviointi syväoppimisessa

Lassi Meronen

Computer Vision and Pattern Recognition · Aaltodoc (Aalto University) · · cited 6× · OpenAlex ↗ · more like this ·

Deep learning has recently become a popular method for solving problems involving large data sets, and in many applications, human performance has been exceeded. However, deep learning models tend to be overconfident in their predictions, especially when encountering new input samples that differ from anything the model has learned during training. This thesis aims to address this problem by developing uncertainty quantification techniques that allow deep learning models to recognise the limits of their capabilities better and when they should be uncertain in their predictions. Improved uncertainty quantification would enable deep learning models to be used in safety-critical applications that require reliable uncertainty estimates. Uncertainty quantification is improved through a Bayesian perspective, and making connections between neural networks and Gaussian processes is at the core…

Periodic Activation Functions Induce Stationarity

Lassi Meronen, Martin Trapp, Arno Solin

cs.LG · · cited 1× · arXiv ↗ · more like this ·

Neural network models are known to reinforce hidden data biases, making them unreliable and difficult to interpret. We seek to build models that `know what they do not know' by introducing inductive biases in the function space. We show that periodic activation functions in Bayesian neural networks establish a connection between the prior on the network weights and translation-invariant, stationary Gaussian process priors. Furthermore, we show that this link goes beyond sinusoidal (Fourier) activations by also covering triangular wave and periodic ReLU activation functions. In a series of experiments, we show that periodic activation functions obtain comparable performance for in-domain data and capture sensitivity to perturbed inputs in deep neural networks for out-of-domain detection.

Deep Learning with Uncertainty Quantification for Emitter Classification

Lassi Meronen

Artificial Intelligence · Aaltodoc (Aalto University) · · cited 0× · OpenAlex ↗ · more like this ·

Emitter classification is a crucial method of target recognition in electronic warfare. The recent popularity and success of deep learning models have made them an appealing method to be applied to emitter classification. However, the lack of reliable uncertainty estimates for deep learning models risks making overconfident incorrect decisions with irreversible consequences. The goal of this thesis is to compare different uncertainty estimation methods for deep learning models, applied to emitter classification tasks. The compared methods are Monte Carlo dropout, deep ensembles, Bayesian neural networks and noise contrastive priors. The neural network structure for all models is a convolutional residual neural network and the emitter classification data is generated through simulations. The results show that uncertainty estimates of Monte Carlo dropout, deep ensembles and Bayesian…

Movement Tracking by Optical Flow Assisted Inertial Navigation

Lassi Meronen, William J. Wilkinson, Arno Solin

cs.CV · · arXiv ↗ · more like this ·

Robust and accurate six degree-of-freedom tracking on portable devices remains a challenging problem, especially on small hand-held devices such as smartphones. For improved robustness and accuracy, complementary movement information from an IMU and a camera is often fused. Conventional visual-inertial methods fuse information from IMUs with a sparse cloud of feature points tracked by the device camera. We consider a visually dense approach, where the IMU data is fused with the dense optical flow field estimated from the camera data. Learning-based methods applied to the full image frames can leverage visual cues and global consistency of the flow field to improve the flow estimates. We show how a learning-based optical flow model can be combined with conventional inertial navigation, and how ideas from probabilistic deep learning can aid the robustness of the measurement updates. The…