arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Daniel Simpson

80 papers · 1977–2026

1977: 11978: 01979: 11980: 01981: 01982: 01983: 01984: 01985: 01986: 11987: 01988: 01989: 01990: 01991: 01992: 01993: 01994: 01995: 11996: 01997: 01998: 01999: 02000: 02001: 02002: 02003: 02004: 02005: 02006: 02007: 02008: 02009: 22010: 02011: 42012: 52013: 112014: 42015: 72016: 42017: 62018: 52019: 72020: 42021: 22022: 12023: 12024: 22025: 02026: 11
1977peak 11 in 20132026

Frequent co-authors

Papers

showing 1–25 of 80 RSS

Rank-normalization, folding, and localization: An improved $\widehat{R}$ for assessing convergence of MCMC

Aki Vehtari, Andrew Gelman, Daniel Simpson +2 more

stat.CO · · cited 1,590× · arXiv ↗ · more like this ·

Markov chain Monte Carlo is a key computational tool in Bayesian statistics, but it can be challenging to monitor the convergence of an iterative stochastic algorithm. In this paper we show that the convergence diagnostic $\widehat{R}$ of Gelman and Rubin (1992) has serious flaws. Traditional $\widehat{R}$ will fail to correctly diagnose convergence failures when the chain has a heavy tail or when the variance varies across the chains. In this paper we propose an alternative rank-based diagnostic that fixes these problems. We also introduce a collection of quantile-based local efficiency measures, along with a practical approach for computing Monte Carlo error estimates for quantiles. We suggest that common trace plots should be replaced with rank plots from multiple chains. Finally, we give recommendations for how these methods should be used in practice.

Penalising model component complexity: A principled, practical approach to constructing priors

Daniel P. Simpson, Håvard Rue, Thiago G. Martins +2 more

stat.ME · · cited 1,148× · arXiv ↗ · more like this ·

In this paper, we introduce a new concept for constructing prior distributions. We exploit the natural nested structure inherent to many model components, which defines the model component to be a flexible extension of a base model. Proper priors are defined to penalise the complexity induced by deviating from the simpler base model and are formulated after the input of a user-defined scaling parameter for that model component, both in the univariate and the multivariate case. These priors are invariant to reparameterisations, have a natural connection to Jeffreys' priors, are designed to support Occam's razor and seem to have excellent robustness properties, all which are highly desirable and allow us to use this approach to define default prior distributions. Through examples and theoretical results, we demonstrate the appropriateness of this approach and how it can be applied in…

Visualization in Bayesian workflow

Jonah Gabry, Daniel Simpson, Aki Vehtari +2 more

stat.ME · · cited 1,025× · arXiv ↗ · more like this ·

Bayesian data analysis is about more than just computing a posterior distribution, and Bayesian visualization is about more than trace plots of Markov chains. Practical Bayesian data analysis, like all data analysis, is an iterative process of model building, inference, model checking and evaluation, and model expansion. Visualization is helpful in each of these stages of the Bayesian workflow and it is indispensable when drawing inferences from the types of modern, high-dimensional models that are used by applied researchers.

An intuitive Bayesian spatial model for disease mapping that accounts for scaling

Andrea Riebler, Sigrunn H. Sørbye, Daniel Simpson +1 more

stat.ME · · cited 522× · arXiv ↗ · more like this ·

In recent years, disease mapping studies have become a routine application within geographical epidemiology and are typically analysed within a Bayesian hierarchical model formulation. A variety of model formulations for the latent level have been proposed but all come with inherent issues. In the classical BYM model, the spatially structured component cannot be seen independently from the unstructured component. This makes prior definitions for the hyperparameters of the two random effects challenging. There are alternative model formulations that address this confounding, however, the issue on how to choose interpretable hyperpriors is still unsolved. Here, we discuss a recently proposed parameterisation of the BYM model that leads to improved parameter control as the hyperparameters can be seen independently from each other. Furthermore, the need for a scaled spatial component is…

The prior can generally only be understood in the context of the likelihood

Andrew Gelman, Daniel Simpson, Michael Betancourt

stat.ME · · cited 500× · arXiv ↗ · more like this ·

A key sticking point of Bayesian analysis is the choice of prior distribution, and there is a vast literature on potential defaults including uniform priors, Jeffreys' priors, reference priors, maximum entropy priors, and weakly informative priors. These methods, however, often manifest a key conceptual tension in prior modeling: a model encoding true prior information should be chosen without reference to the model of the measurement process, but almost all common prior modeling techniques are implicitly motivated by a reference likelihood. In this paper we resolve this apparent paradox by placing the choice of prior into the context of the entire Bayesian analysis, from inference to prediction to model evaluation.

The Prior Can Often Only Be Understood in the Context of the Likelihood

Andrew Gelman, Daniel Simpson, Michael Betancourt

Statistics and Probability · Columbia Academic Commons (Columbia University) · · cited 464× · DOI ↗ · more like this ·

A key sticking point of Bayesian analysis is the choice of prior distribution, and there is a vast literature on potential defaults including uniform priors, Jeffreys’ priors, reference priors, maximum entropy priors, and weakly informative priors. These methods, however, often manifest a key conceptual tension in prior modeling: a model encoding true prior information should be chosen without reference to the model of the measurement process, but almost all common prior modeling techniques are implicitly motivated by a reference likelihood. In this paper we resolve this apparent paradox by placing the choice of prior into the context of the entire Bayesian analysis, from inference to prediction to model evaluation.

Using stacking to average Bayesian predictive distributions

Yuling Yao, Aki Vehtari, Daniel Simpson +1 more

stat.ME · · cited 441× · arXiv ↗ · more like this ·

The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit. We take the idea of stacking from the point estimation literature and generalize to the combination of predictive distributions, extending the utility function to any proper scoring rule, using Pareto smoothed importance sampling to efficiently compute the required leave-one-out posterior distributions and regularization to get more stability. We compare stacking of predictive distributions to several alternatives: stacking of means, Bayesian model averaging (BMA), pseudo-BMA using AIC-type weighting, and a variant of pseudo-BMA that is stabilized using the Bayesian bootstrap. Based on simulations and real-data applications, we recommend stacking of predictive distributions, with BB-pseudo-BMA as an…

Validating Bayesian Inference Algorithms with Simulation-Based Calibration

Sean Talts, Michael Betancourt, Daniel Simpson +2 more

stat.ME · · cited 438× · arXiv ↗ · more like this ·

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This procedure not only identifies inaccurate computation and inconsistencies in model implementations but also provides graphical summaries that can indicate the nature of the problems that arise. We argue that SBC is a critical part of a robust Bayesian workflow, as well as being a useful tool for those developing computational algorithms and statistical software.

Bayesian Workflow

Andrew Gelman, Aki Vehtari, Daniel Simpson +7 more

stat.ME · · cited 367× · arXiv ↗ · more like this ·

The Bayesian approach to data analysis provides a powerful way to handle uncertainty in all observations, model parameters, and model structure using probability theory. Probabilistic programming languages make it easier to specify and fit Bayesian models, but this still leaves us with many options regarding constructing, evaluating, and using these models, along with many remaining challenges in computation. Using Bayesian inference to solve real-world problems requires not only statistical skills, subject matter knowledge, and programming, but also awareness of the decisions made in the process of data analysis. All of these aspects can be understood as part of a tangled workflow of applied Bayesian statistics. Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model…

Yes, but Did It Work?: Evaluating Variational Inference

Yuling Yao, Aki Vehtari, Daniel Simpson +1 more

stat.ML · · cited 165× · arXiv ↗ · more like this ·

While it's always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement for joint distributions, while simultaneously improving the error in the estimate. The variational simulation-based calibration (VSBC) assesses the average performance of point estimates.

On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods

Anne-Marie Lyne, Mark Girolami, Yves Atchadé +2 more

stat.ME · · cited 150× · arXiv ↗ · more like this ·

A large number of statistical models are "doubly-intractable": the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (MCMC), cannot be used. Examples include, but are not confined to, massive Gaussian Markov random fields, autologistic models and Exponential random graph models. A number of approximate schemes based on MCMC techniques, Approximate Bayesian computation (ABC) or analytic approximations to the posterior have been suggested, and these are reviewed here. Exact MCMC schemes, which can be applied to a subset of doubly-intractable distributions, have also been developed and are described in this paper. As yet, no general method exists which can be applied to all classes of models…

Gaydar and the Fallacy of Decontextualized Measurement

Andrew Gelman, Greggor Mattson, Daniel Simpson

Anthropology · Sociological Science · · cited 53× · DOI ↗ · more like this ·

Recent media coverage of studies about 'gaydar,' the supposed ability to detect another's sexual orientation through visual cues, reveal problems in which the ideals of scientific precision strip the context from intrinsically social phenomena. This fallacy of objective measurement, as we term it, leads to nonsensical claims based on the predictive accuracy of statistical significance. We interrogate these gaydar studies' assumption that there is some sort of pure biological measure of perception of sexual orientation. Instead, we argue that the concept of gaydar inherently exists within a social context and that this should be recognized when studying it. We use this case as an example of a more general concern about illusory precision in the measurement of social phenomena and suggest statistical strategies to address common problems.

Improving multilevel regression and poststratification with structured priors

Yuxiang Gao, Lauren Kennedy, Daniel Simpson +1 more

stat.ME · · cited 51× · arXiv ↗ · more like this ·

A central theme in the field of survey statistics is estimating population-level quantities through data coming from potentially non-representative samples of the population. Multilevel Regression and Poststratification (MRP), a model-based approach, is gaining traction against the traditional weighted approach for survey estimates. MRP estimates are susceptible to bias if there is an underlying structure that the methodology does not capture. This work aims to provide a new framework for specifying structured prior distributions that lead to bias reduction in MRP estimates. We use simulation studies to explore the benefit of these prior distributions and demonstrate their efficacy on non-representative US survey data. We show that structured prior distributions offer absolute bias reduction and variance reduction for posterior MRP estimates in a large variety of data regimes.

Asynchronous Gibbs Sampling

Alexander Terenin, Daniel Simpson, David Draper

stat.CO · · cited 49× · arXiv ↗ · more like this ·

Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method often used in Bayesian learning. MCMC methods can be difficult to deploy on parallel and distributed systems due to their inherently sequential nature. We study asynchronous Gibbs sampling, which achieves parallelism by simply ignoring sequential requirements. This method has been shown to produce good empirical results for some hierarchical models, and is popular in the topic modeling community, but was also shown to diverge for other targets. We introduce a theoretical framework for analyzing asynchronous Gibbs sampling and other extensions of MCMC that do not possess the Markov property. We prove that asynchronous Gibbs can be modified so that it converges under appropriate regularity conditions -- we call this the exact asynchronous Gibbs algorithm. We study asynchronous Gibbs on a set of examples by comparing the exact and…

Fast approximate inference with INLA: the past, the present and the future

Daniel Simpson, Finn Lindgren, Håvard Rue

stat.CO · · cited 21× · arXiv ↗ · more like this ·

Latent Gaussian models are an extremely popular, flexible class of models. Bayesian inference for these models is, however, tricky and time consuming. Recently, Rue, Martino and Chopin introduced the Integrated Nested Laplace Approximation (INLA) method for deterministic fast approximate inference. In this paper, we outline the INLA approximation and its related R package. We will discuss the newer components of the r-INLA program as well as some possible extensions.

LGM Split Sampler: An Efficient MCMC Sampling Scheme for Latent Gaussian Models

Óli Páll Geirsson, Birgir Hrafnkelsson, Daniel Simpson +1 more

Artificial Intelligence · Statistical Science · · cited 13× · DOI ↗ · more like this ·

A general and flexible class of latent Gaussian models is proposed in this paper. The latent Gaussian model is adapted to the generalized additive model for location, scale and shape (GAMLSS), that is, the data density function of each data point can depend on more than a single linear predictor of the latent parameters. We refer to this framework as extended latent Gaussian models. The most commonly applied latent Gaussian models (LGMs) are such that a linear predictor is proposed only for the location parameter. Extended LGMs allow proposing linear predictors also for the scale parameter and potentially other parameters. We propose a novel computationally efficient Markov chain Monte Carlo sampling scheme for the extended LGMs which we refer to as the LGM split sampler. It is a two block Gibbs sampling scheme designed to exploit the model structure of the extended LGMs. An extended…

The Complexity of Pebbling in Diameter Two Graphs

Charles A. Cusack, Timothy Lewis, Daniel Simpson +1 more

Computational Theory and Mathematics · SIAM Journal on Discrete Mathematics · · cited 10× · DOI ↗ · more like this ·

Given a simple, connected graph, a pebbling configuration is a function from its vertex set to the nonnegative integers. A pebbling move between adjacent vertices removes two pebbles from one vertex and adds one pebble to the other. A vertex $r$ is said to be reachable from a configuration if there exists a sequence of pebbling moves that places one pebble on $r$. A configuration is solvable if every vertex is reachable. We prove tight bounds on the number of vertices with two and three pebbles that an unsolvable configuration on a diameter two graph can have in terms of the size of the graph. We also prove that determining reachability of a vertex is NP-complete, even in graphs of diameter two.

Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation: Bayesian inference for latent Gaussian models and beyond

Charles C. Margossian, Aki Vehtari, Daniel Simpson +1 more

stat.CO · · cited 9× · arXiv ↗ · more like this ·

Gaussian latent variable models are a key class of Bayesian hierarchical models with applications in many fields. Performing Bayesian inference on such models can be challenging as Markov chain Monte Carlo algorithms struggle with the geometry of the resulting posterior distribution and can be prohibitively slow. An alternative is to use a Laplace approximation to marginalize out the latent Gaussian variables and then integrate out the remaining hyperparameters using dynamic Hamiltonian Monte Carlo, a gradient-based Markov chain Monte Carlo sampler. To implement this scheme efficiently, we derive a novel adjoint method that propagates the minimal information needed to construct the gradient of the approximate marginal likelihood. This strategy yields a scalable differentiation method that is orders of magnitude faster than state of the art differentiation techniques when the…

The Integrated Nested Laplace Approximation for fitting Dirichlet regression models

Joaquín Martínez-Minaya, Finn Lindgren, Antonio López-Quílez +2 more

stat.CO · · cited 9× · arXiv ↗ · more like this ·

This paper introduces a Laplace approximation to Bayesian inference in Dirichlet regression models, which can be used to analyze a set of variables on a simplex exhibiting skewness and heteroscedasticity, without having to transform the data. These data, which mainly consist of proportions or percentages of disjoint categories, are widely known as compositional data and are common in areas such as ecology, geology, and psychology. We provide both the theoretical foundations and a description of how Laplace approximation can be implemented in the case of Dirichlet regression. The paper also introduces the package dirinla in the R-language that extends the R-INLA package, which can not deal directly with Dirichlet likelihoods. Simulation studies are presented to validate the good behaviour of the proposed method, while a real data case-study is used to show how this approach can be…

Scalable iterative methods for sampling from massive Gaussian random vectors

Daniel P. Simpson, Ian W. Turner, Christopher M. Strickland +1 more

stat.CO · · cited 7× · arXiv ↗ · more like this ·

Sampling from Gaussian Markov random fields (GMRFs), that is multivariate Gaussian ran- dom vectors that are parameterised by the inverse of their covariance matrix, is a fundamental problem in computational statistics. In this paper, we show how we can exploit arbitrarily accu- rate approximations to a GMRF to speed up Krylov subspace sampling methods. We also show that these methods can be used when computing the normalising constant of a large multivariate Gaussian distribution, which is needed for both any likelihood-based inference method. The method we derive is also applicable to other structured Gaussian random vectors and, in particu- lar, we show that when the precision matrix is a perturbation of a (block) circulant matrix, it is still possible to derive O(n log n) sampling schemes.

Combining Opinions for Use in Bayesian Networks: A Measurement Error Approach

Anna Charisse Farr, Kerrie Mengersen, Fabrizio Ruggeri +3 more

Artificial Intelligence · International Statistical Review · · cited 6× · DOI ↗ · more like this ·

Summary Bayesian networks (BNs) are graphical probabilistic models used for reasoning under uncertainty. These models are becoming increasingly popular in a range of fields including engineering, ecology, computational biology, medical diagnosis and forensics. In most of these cases, the BNs are quantified using information from experts or from users' opinions. While this quantification is straightforward for one expert, there is still debate about how to represent opinions from multiple experts in a BN. This paper proposes the use of a measurement error model to achieve this. The proposed model addresses the issues associated with current methods of combining opinions such as the absence of a coherent probability model, the loss of the conditional independence structure of the BN and the provision of only a point estimate for the consensus. The proposed model is applied to a subnetwork…

Careful prior specification avoids incautious inference for log-Gaussian Cox point processes

Sigrunn H. Sørbye, Janine B. Illian, Daniel P. Simpson +1 more

stat.ME · · cited 6× · arXiv ↗ · more like this ·

Prior specifications for hyperparameters of random fields in Bayesian spatial point process modelling can have a major impact on the statistical inference and the conclusions made. We consider fitting of log-Gaussian Cox processes to spatial point patterns relative to spatial covariate data. From an ecological point of view, an important aim of the analysis is to assess significant associations between the covariates and the point pattern intensity of a given species. This paper introduces the use of a reparameterised model to facilitate meaningful interpretations of the results and how these depend on hyperprior specifications. The model combines a scaled spatially structured field with an unstructured random field, having a common precision parameter. An additional hyperparameter identifies the fraction of variance explained by the spatially structured term and proper scaling makes…

Explainable Deep Learning Models for Dynamic and Online Malware Classification

Quincy Card, Daniel Simpson, Kshitiz Aryal +2 more

cs.CR · · cited 4× · arXiv ↗ · more like this ·

In recent years, there has been a significant surge in malware attacks, necessitating more advanced preventive measures and remedial strategies. While several successful AI-based malware classification approaches exist categorized into static, dynamic, or online analysis, most successful AI models lack easily interpretable decisions and explanations for their processes. Our paper aims to delve into explainable malware classification across various execution environments (such as dynamic and online), thoroughly analyzing their respective strengths, weaknesses, and commonalities. To evaluate our approach, we train Feed Forward Neural Networks (FFNN) and Convolutional Neural Networks (CNN) to classify malware based on features obtained from dynamic and online analysis environments. The feature attribution for malware classification is performed by explainability tools, SHAP, LIME and…

You Just Keep on Pushing My Love over the Borderline: A Rejoinder

Daniel Simpson, Håvard Rue, Andrea Riebler +2 more

Sociology and Political Science · Statistical Science · · cited 4× · DOI ↗ · more like this ·

<i>INTRODUCTION</i>: The point of departure for our paper is that most modern statistical models are built to be flexible enough to model diverse data generating mechanisms. Good statistical practice requires us to limit this flexibility, which is typically controlled by a small number of parameters, to the amount “needed” to model the data at hand. The Bayesian framework provides a natural method for doing this although, as DD points out, this trend for penalising model complexity casts a broad shadow over all of modern statistics and data science.The PC prior framework argues for setting priors on these flexibility parameters that are specifically built to penalise a certain type of complexity and avoid overfitting. The discussants raised various points about this core idea. First, DD pointed out that while over-fitting a model is a bad thing, under-fitting is not better…