arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Alex Tank

12 papers · 2014–2022

2014: 12015: 12016: 22017: 52018: 12019: 02020: 02021: 12022: 1
2014peak 5 in 20172022

Frequent co-authors

Papers

showing 1–12 of 12 RSS

Neural Granger Causality

Alex Tank, Ian Covert, Nicholas Foti +2 more

stat.ML · · cited 107× · arXiv ↗ · more like this ·

While most classical approaches to Granger causality detection assume linear dynamics, many interactions in real-world applications, like neuroscience and genomics, are inherently nonlinear. In these cases, using linear models may lead to inconsistent estimation of Granger causal interactions. We propose a class of nonlinear methods by applying structured multilayer perceptrons (MLPs) or recurrent neural networks (RNNs) combined with sparsity-inducing penalties on the weights. By encouraging specific sets of weights to be zero--in particular, through the use of convex group-lasso penalties--we can extract the Granger causal structure. To further contrast with traditional approaches, our framework naturally enables us to efficiently capture long-range dependencies between series either via our RNNs or through an automatic lag selection in the MLP. We show that our neural Granger…

An Interpretable and Sparse Neural Network Model for Nonlinear Granger Causality Discovery

Alex Tank, Ian Cover, Nicholas J. Foti +2 more

stat.ML · · cited 26× · arXiv ↗ · more like this ·

While most classical approaches to Granger causality detection repose upon linear time series assumptions, many interactions in neuroscience and economics applications are nonlinear. We develop an approach to nonlinear Granger causality detection using multilayer perceptrons where the input to the network is the past time lags of all series and the output is the future value of a single series. A sufficient condition for Granger non-causality in this setting is that all of the outgoing weights of the input data, the past lags of a series, to the first hidden layer are zero. For estimation, we utilize a group lasso penalty to shrink groups of input weights to zero. We also propose a hierarchical penalty for simultaneous Granger causality and lag estimation. We validate our approach on simulated data from both a sparse linear autoregressive model and the sparse and nonlinear Lorenz-96…

Bayesian Structure Learning for Stationary Time Series

Alex Tank, Nicholas Foti, Emily Fox

stat.ME · · cited 26× · arXiv ↗ · more like this ·

While much work has explored probabilistic graphical models for independent data, less attention has been paid to time series. The goal in this setting is to determine conditional independence relations between entire time series, which for stationary series, are encoded by zeros in the inverse spectral density matrix. We take a Bayesian approach to structure learning, placing priors on (i) the graph structure and (ii) spectral matrices given the graph. We leverage a Whittle likelihood approximation and define a conjugate prior---the hyper complex inverse Wishart---on the complex-valued and graph-constrained spectral matrices. Due to conjugacy, we can analytically marginalize the spectral matrices and obtain a closed-form marginal likelihood of the time series given a graph. Importantly, our analytic marginal likelihood allows us to avoid inference of the complex spectral matrices…

Granger Causality Networks for Categorical Time Series

Alex Tank, Emily B. Fox, Ali Shojaie

stat.ME · · cited 20× · arXiv ↗ · more like this ·

We present a new framework for learning Granger causality networks for multivariate categorical time series, based on the mixture transition distribution (MTD) model. Traditionally, MTD is plagued by a nonconvex objective, non-identifiability, and presence of many local optima. To circumvent these problems, we recast inference in the MTD as a convex problem. The new formulation facilitates the application of MTD to high-dimensional multivariate time series. As a baseline, we also formulate a multi-output logistic autoregressive model (mLTD), which while a straightforward extension of autoregressive Bernoulli generalized linear models, has not been previously applied to the analysis of multivariate categorial time series. We develop novel identifiability conditions of the MTD model and compare them to those for mLTD. We further devise novel and efficient optimization algorithm for the…

The Convex Mixture Distribution: Granger Causality for Categorical Time Series

Alex Tank, Xiudi Li, Emily B. Fox +1 more

Statistics and Probability · SIAM Journal on Mathematics of Data Science · · cited 10× · DOI ↗ · more like this ·

We present a framework for learning Granger causality networks for multivariate categorical time series based on the mixture transition distribution (MTD) model. Traditionally, MTD is plagued by a nonconvex objective, non-identifiability, and presence of local optima. To circumvent these problems, we recast inference in the MTD as a convex problem. The new formulation facilitates the application of MTD to high-dimensional multivariate time series. As a baseline, we also formulate a multi-output logistic autoregressive model (mLTD), which while a straightforward extension of autoregressive Bernoulli generalized linear models, has not been previously applied to the analysis of multivariate categorial time series. We establish identifiability conditions of the MTD model and compare them to those for mLTD. We further devise novel and efficient optimization algorithms for MTD based on our…

A Unified Framework for Long Range and Cold Start Forecasting of Seasonal Profiles in Time Series

Christopher Xie, Alex Tank, Alec Greaves-Tunnell +1 more

stat.ML · · cited 8× · arXiv ↗ · more like this ·

Providing long-range forecasts is a fundamental challenge in time series modeling, which is only compounded by the challenge of having to form such forecasts when a time series has never previously been observed. The latter challenge is the time series version of the cold-start problem seen in recommender systems which, to our knowledge, has not been addressed in previous work. A similar problem occurs when a long range forecast is required after only observing a small number of time points --- a warm start forecast. With these aims in mind, we focus on forecasting seasonal profiles---or baseline demand---for periods on the order of a year in three cases: the long range case with multiple previously observed seasonal profiles, the cold start case with no previous observed seasonal profiles, and the warm start case with only a single partially observed profile. Classical time series…

Streaming Variational Inference for Bayesian Nonparametric Mixture Models

Alex Tank, Nicholas J. Foti, Emily B. Fox

stat.ML · · cited 8× · arXiv ↗ · more like this ·

In theory, Bayesian nonparametric (BNP) models are well suited to streaming data scenarios due to their ability to adapt model complexity with the observed data. Unfortunately, such benefits have not been fully realized in practice; existing inference algorithms are either not applicable to streaming applications or not extensible to BNP models. For the special case of Dirichlet processes, streaming inference has been considered. However, there is growing interest in more flexible BNP models building on the class of normalized random measures (NRMs). We work within this general framework and present a streaming variational inference algorithm for NRM mixture models. Our algorithm is based on assumed density filtering (ADF), leading straightforwardly to expectation propagation (EP) for large-scale batch inference as well. We demonstrate the efficacy of the algorithm on clustering…

Penalized estimation of threshold auto-regressive models with many components and thresholds

Kunhui Zhang, Abolfazl Safikhani, Alex Tank +1 more

Statistics and Probability · Electronic Journal of Statistics · · cited 4× · DOI ↗ · more like this ·

Thanks to their simplicity and interpretable structure, autoregressive processes are widely used to model time series data. However, many real time series data sets exhibit non-linear patterns, requiring nonlinear modeling. The threshold Auto-Regressive (TAR) process provides a family of non-linear auto-regressive time series models in which the process dynamics are specific step functions of a thresholding variable. While estimation and inference for low-dimensional TAR models have been investigated, high-dimensional TAR models have received less attention. In this article, we develop a new framework for estimating high-dimensional TAR models, and propose two different sparsity-inducing penalties. The first penalty corresponds to a natural extension of classical TAR model to high-dimensional settings, where the same threshold is enforced for all model parameters. Our second penalty…

An Efficient ADMM Algorithm for Structural Break Detection in Multivariate Time Series

Alex Tank, Emily B. Fox, Ali Shojaie

stat.ML · · cited 3× · arXiv ↗ · more like this ·

We present an efficient alternating direction method of multipliers (ADMM) algorithm for segmenting a multivariate non-stationary time series with structural breaks into stationary regions. We draw from recent work where the series is assumed to follow a vector autoregressive model within segments and a convex estimation procedure may be formulated using group fused lasso penalties. Our ADMM approach first splits the convex problem into a global quadratic program and a simple group lasso proximal update. We show that the global problem may be parallelized over rows of the time dependent transition matrices and furthermore that each subproblem may be rewritten in a form identical to the log-likelihood of a Gaussian state space model. Consequently, we develop a Kalman smoothing algorithm to solve the global update in time linear in the length of the series.

Supplemental Information: Streaming Variational Inference for Bayesian Nonparametric Mixture Models

Alex Tank, Nicholas J. Foti, Emily B. Fox

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

This section motivates the use of the mean field update for the global variables, given in Eq. (19) of the main text, as an approximation to the optimal update for ADF. The presentation adapts that of [1] for ADF-NRM. Let p̂(θ, z1:n|x1:n) ∝ p(xn|θ, zn)p(zn|z1:n)q̂(θ, z1:n−1) denote the approximate posterior under the past variational updates after adding in the nth observation. The optimal q(θk) under ADF is given by the marginal distribution of p̂: q̂n(θk) ∝ z1:n p̂(θk|θ\\k, z1:n, x1:n)p̂(θ\\k|z1:n, x1:n)p̂(z1:n|x1:n)dθ\\k. (1) Both the sums and integrals are intractable so we use the approximations: p̂(θ\\k|z1:n, x1:n) ≈ q̂n(θ\\k) and p̂(z1:n|x1:n) ≈ q̂n(z1:n) = ∏n i=1 q̂n(zi) which yields: q̂n(θk) z1:n p̂(θk|θ\\k, z1:n, x1:n)q̂n(θ\\k)q̂n(z1:n)dθ\\k (2)

Streaming Variational Inference for Bayesian Nonparametric Mixture Models. arXiv.org

Alex Tank, Nicholas J. Foti, Emily B. Fox

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

In theory, Bayesian nonparametric (BNP) models are well suited to streaming data sce-narios due to their ability to adapt model complexity with the observed data. Unfor-tunately, such benefits have not been fully realized in practice; existing inference algo-rithms are either not applicable to stream-ing applications or not extensible to BNP models. For the special case of Dirichlet processes, streaming inference has been con-sidered. However, there is growing inter-est in more flexible BNP models building on the class of normalized random measures (NRMs). We work within this general frame-work and present a streaming variational in-ference algorithm for NRM mixture models. Our algorithm is based on assumed density filtering (ADF), leading straightforwardly to expectation propagation (EP) for large-scale batch inference as well. We demonstrate the efficacy of the algorithm on clustering…

Identifiability and Estimation of Structural Vector Autoregressive Models for Subsampled and Mixed Frequency Time Series

Alex Tank, Emily B. Fox, Ali Shojaie

stat.ME · · arXiv ↗ · more like this ·

Causal inference in multivariate time series is challenging due to the fact that the sampling rate may not be as fast as the timescale of the causal interactions. In this context, we can view our observed series as a subsampled version of the desired series. Furthermore, due to technological and other limitations, series may be observed at different sampling rates, representing a mixed frequency setting. To determine instantaneous and lagged effects between time series at the true causal scale, we take a model-based approach based on structural vector autoregressive (SVAR) models. In this context, we present a unifying framework for parameter identifiability and estimation under both subsampling and mixed frequencies when the noise, or shocks, are non-Gaussian. Importantly, by studying the SVAR case, we are able to both provide identifiability and estimation methods for the causal…