arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Timo Dimitriadis

21 papers · 2017–2026

2017: 12018: 12019: 22020: 32021: 12022: 62023: 22024: 32025: 02026: 2
2017peak 6 in 20222026

Frequent co-authors

Papers

showing 1–21 of 21 RSS

Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisited

Timo Dimitriadis, Tilmann Gneiting, Alexander I. Jordan

stat.ME · · cited 108× · arXiv ↗ · more like this ·

A probability forecast or probabilistic classifier is reliable or calibrated if the predicted probabilities are matched by ex post observed frequencies, as examined visually in reliability diagrams. The classical binning and counting approach to plotting reliability diagrams has been hampered by a lack of stability under unavoidable, ad hoc implementation decisions. Here we introduce the CORP approach, which generates provably statistically Consistent, Optimally binned, and Reproducible reliability diagrams in an automated way. CORP is based on non-parametric isotonic regression and implemented via the Pool-adjacent-violators (PAV) algorithm - essentially, the CORP reliability diagram shows the graph of the PAV- (re)calibrated forecast probabilities. The CORP approach allows for uncertainty quantification via either resampling techniques or asymptotic theory, furnishes a new numerical…

Honest calibration assessment for binary outcome predictions

Timo Dimitriadis, Lutz Duembgen, Alexander Henzi +2 more

math.ST · · cited 24× · arXiv ↗ · more like this ·

Probability predictions from binary regressions or machine learning methods ought to be calibrated: If an event is predicted to occur with probability $x$, it should materialize with approximately that frequency, which means that the so-called calibration curve $p(\cdot)$ should equal the identity, $p(x) = x$ for all $x$ in the unit interval. We propose honest calibration assessment based on novel confidence bands for the calibration curve, which are valid only subject to the natural assumption of isotonicity. Besides testing the classical goodness-of-fit null hypothesis of perfect calibration, our bands facilitate inverted goodness-of-fit tests whose rejection allows for the sought-after conclusion of a sufficiently well specified model. We show that our bands have a finite sample coverage guarantee, are narrower than existing approaches, and adapt to the local smoothness of the…

Osband's Principle for Identification Functions

Timo Dimitriadis, Tobias Fissler, Johanna Ziegel

math.ST · · cited 6× · arXiv ↗ · more like this ·

Given a statistical functional of interest such as the mean or median, a (strict) identification function is zero in expectation at (and only at) the true functional value. Identification functions are key objects in forecast validation, statistical estimation and dynamic modelling. For a possibly vector-valued functional of interest, we fully characterise the class of (strict) identification functions subject to mild regularity conditions.

A Joint Quantile and Expected Shortfall Regression Framework

Timo Dimitriadis, Sebastian Bayer

math.ST · · cited 4× · arXiv ↗ · more like this ·

We introduce a novel regression framework which simultaneously models the quantile and the Expected Shortfall (ES) of a response variable given a set of covariates. This regression is based on a strictly consistent loss function for the pair quantile and ES, which allows for M- and Z-estimation of the joint regression parameters. We show consistency and asymptotic normality for both estimators under weak regularity conditions. The underlying loss function depends on two specification functions, whose choice affects the properties of the resulting estimators. We find that the Z-estimator is numerically unstable and thus, we rely on M-estimation of the model parameters. Extensive simulations verify the asymptotic properties and analyze the small sample behavior of the M-estimator for different specification functions. This joint regression framework allows for various applications…

The Efficiency Gap

Timo Dimitriadis, Tobias Fissler, Johanna Ziegel

math.ST · · cited 2× · arXiv ↗ · more like this ·

Parameter estimation via M- and Z-estimation is equally powerful in semiparametric models for one-dimensional functionals due to a one-to-one relation between corresponding loss and identification functions via integration and differentiation. For multivariate functionals such as multiple moments, quantiles, or the pair (Value at Risk, Expected Shortfall), this one-to-one relation fails and not every identification function possesses an antiderivative. The most important implication is an efficiency gap: The most efficient Z-estimator often outperforms the most efficient M-estimator. We theoretically establish this phenomenon for multiple quantiles at different levels and for the pair (Value at Risk, Expected Shortfall), and illustrate the gap numerically. Our results further give guidance for pseudo-efficient M-estimation for semiparametric models of the Value at Risk and Expected…

Expected Kullback-Leibler-based characterizations of score-driven updates

Ramon de Punder, Timo Dimitriadis, Rutger-Jan Lange

math.ST · · cited 1× · arXiv ↗ · more like this ·

Score-driven (SD) models are a standard tool in statistics and econometrics, with applications in hundreds of published articles in the past decade. We provide an information-theoretic characterization of SD updates based on reductions in the expected Kullback-Leibler (EKL) divergence relative to the true -- but unknown -- data-generating density. EKL reductions occur if and only if the expected update direction aligns with the expected score; i.e., their inner product should be positive. This equivalence condition uniquely identifies SD updates (including scaled or clipped variants) as being EKL reducing, even in non-concave, multivariate, and misspecified settings. We further derive explicit bounds on admissible learning rates in terms of score moments, linking SD methods to adaptive optimization techniques. By contrast, alternative performance measures in the literature impose…

Dynamic CoVaR Modeling and Estimation

Timo Dimitriadis, Yannick Hoga

econ.EM · · cited 1× · arXiv ↗ · more like this ·

The popular systemic risk measure CoVaR (conditional Value-at-Risk) and its variants are widely used in economics and finance. In this article, we propose joint dynamic forecasting models for the Value-at-Risk (VaR) and CoVaR. The CoVaR version we consider is defined as a large quantile of one variable (e.g., losses in the financial system) conditional on some other variable (e.g., losses in a bank's shares) being in distress. We introduce a two-step M-estimator for the model parameters drawing on recently proposed bivariate scoring functions for the pair (VaR, CoVaR). We prove consistency and asymptotic normality of our parameter estimator and analyze its finite-sample properties in simulations. Finally, we apply a specific subclass of our dynamic forecasting models, which we call CoCAViaR models, to log-returns of large US banks. A formal forecast comparison shows that our CoCAViaR…

Shift-Dispersion Decompositions of Wasserstein and Cramér Distances

Johannes Resin, Daniel Wolffram, Johannes Bracher +1 more

stat.ME · · cited 0× · arXiv ↗ · more like this ·

Divergence functions are measures of distance or dissimilarity between probability distributions that serve various purposes in statistics and applications. We propose decompositions of Wasserstein and Cramér distances$-$which compare two distributions by integrating over their differences in distribution or quantile functions$-$into directed shift and dispersion components. These components are obtained by dividing the differences between the quantile functions into contributions arising from shift and dispersion, respectively. Our decompositions add information on how the distributions differ in a condensed form and consequently enhance the interpretability of the underlying divergences. We show that our decompositions satisfy a number of natural properties and are unique in doing so in location-scale families. The decompositions allow to derive sensitivities of the divergence…

Measuring Dependence between Events

Marc-Oliver Pohle, Timo Dimitriadis, Jan-Lukas Wermuth

stat.ME · · cited 0× · arXiv ↗ · more like this ·

Measuring dependence between two events, or equivalently between two binary random variables, amounts to expressing the dependence structure inherent in a $2\times 2$ contingency table in a real number between $-1$ and $1$. Countless such dependence measures exist, but there is little theoretical guidance on how they compare and on their advantages and shortcomings. Thus, practitioners might be overwhelmed by the problem of choosing a suitable measure. We provide a set of natural desirable properties that a proper dependence measure should fulfill. We show that Yule's Q and the little-known Cole coefficient are proper, while the most widely-used measures, the phi coefficient and all contingency coefficients, are improper. They have a severe attainability problem, that is, even under perfect dependence they can be very far away from $-1$ and $1$, and often differ substantially from the…

Characterizing M-estimators

Timo Dimitriadis, Tobias Fissler, Johanna Ziegel

math.ST · · cited 0× · arXiv ↗ · more like this ·

We characterize the full classes of M-estimators for semiparametric models of general functionals by formally connecting the theory of consistent loss functions from forecast evaluation with the theory of M-estimation. This novel characterization result opens up the possibility for theoretical research on efficient and equivariant M-estimation and, more generally, it allows to leverage existing results on loss functions known from the literature of forecast evaluation in estimation theory.

Statistical Inference for Score Decompositions

Timo Dimitriadis, Marius Puke

econ.EM · · arXiv ↗ · more like this ·

We introduce inference methods for score decompositions, which partition scoring functions for predictive assessment into three interpretable components: miscalibration, discrimination, and uncertainty. Our estimation and inference relies on a linear recalibration of the forecasts, which is applicable to general multi-step ahead point forecasts such as means and quantiles due to its validity for both smooth and non-smooth scoring functions. This approach ensures desirable finite-sample properties, enables asymptotic inference, and establishes a direct connection to the classical Mincer-Zarnowitz regression. The resulting inference framework facilitates tests for equal forecast calibration or discrimination, which yield three key advantages. They enhance the information content of predictive ability tests by decomposing scores, deliver higher statistical power in certain scenarios, and…

Systemic Risk Surveillance

Timo Dimitriadis, Yannick Hoga

econ.EM · · arXiv ↗ · more like this ·

Following several episodes of financial market turmoil in recent decades, changes in systemic risk have drawn growing attention. Therefore, we propose surveillance schemes for systemic risk, which allow to detect misspecified systemic risk forecasts in an "online" fashion. This enables daily monitoring of the forecasts while controlling for the accumulation of false test rejections. Such online schemes are vital in taking timely countermeasures to avoid financial distress. Our monitoring procedures allow multiple series at once to be monitored, thus increasing the likelihood and the speed at which early signs of trouble may be picked up. The tests hold size by construction, such that the null of correct systemic risk assessments is only rejected during the monitoring period with (at most) a pre-specified probability. Monte Carlo simulations illustrate the good finite-sample properties…

Regressions under Adverse Conditions

Timo Dimitriadis, Yannick Hoga

econ.EM · · arXiv ↗ · more like this ·

We introduce a new regression method that relates the mean of an outcome variable to covariates, under the "adverse condition" that a distress variable falls in its tail. This allows to tailor classical mean regressions to adverse scenarios, which receive increasing interest in economics and finance, among many others. In the terminology of the systemic risk literature, our method can be interpreted as a regression for the Marginal Expected Shortfall. We propose a two-step procedure to estimate the new models, show consistency and asymptotic normality of the estimator, and propose feasible inference under weak conditions that allow for cross-sectional and time series applications. Simulations verify the accuracy of the asymptotic approximations of the two-step estimator. Two empirical applications show that our regressions under adverse conditions are a valuable tool in such diverse…

Evaluating Probabilistic Classifiers: The Triptych

Timo Dimitriadis, Tilmann Gneiting, Alexander I. Jordan +1 more

stat.ME · · arXiv ↗ · more like this ·

Probability forecasts for binary outcomes, often referred to as probabilistic classifiers or confidence scores, are ubiquitous in science and society, and methods for evaluating and comparing them are in great demand. We propose and study a triptych of diagnostic graphics that focus on distinct and complementary aspects of forecast performance: The reliability diagram addresses calibration, the receiver operating characteristic (ROC) curve diagnoses discrimination ability, and the Murphy diagram visualizes overall predictive performance and value. A Murphy curve shows a forecast's mean elementary scores, including the widely used misclassification rate, and the area under a Murphy curve equals the mean Brier score. For a calibrated forecast, the reliability curve lies on the diagonal, and for competing calibrated forecasts, the ROC and Murphy curves share the same number of crossing…

Efficient Sampling for Realized Variance Estimation in Time-Changed Diffusion Models

Timo Dimitriadis, Roxana Halbleib, Jeannine Polivka +3 more

econ.EM · · arXiv ↗ · more like this ·

This paper analyzes the benefits of sampling intraday returns in intrinsic time for the realized variance (RV) estimator. We theoretically show in finite samples that depending on the permitted sampling information, the RV estimator is most efficient under either hitting time sampling that samples whenever the price changes by a pre-determined threshold, or under the new concept of realized business time that samples according to a combination of observed trades and estimated tick variance. The analysis builds on the assumption that asset prices follow a diffusion that is time-changed with a jump process that separately models the transaction times. This provides a flexible model that allows for leverage specifications and Hawkes-type jump processes and separately captures the empirically varying trading intensity and tick variance processes, which are particularly relevant for…

A safe Hosmer-Lemeshow test

Alexander Henzi, Marius Puke, Timo Dimitriadis +1 more

stat.ME · · arXiv ↗ · more like this ·

This article proposes an alternative to the Hosmer-Lemeshow (HL) test for evaluating the calibration of probability forecasts for binary events. The approach is based on e-values, a new tool for hypothesis testing. An e-value is a random variable with expected value less or equal to one under a null hypothesis. Large e-values give evidence against the null hypothesis, and the multiplicative inverse of an e-value is a p-value. Our test uses online isotonic regression to estimate the calibration curve as a `betting strategy' against the null hypothesis. We show that the test has power against essentially all alternatives, which makes it theoretically superior to the HL test and at the same time resolves the well-known instability problem of the latter. A simulation study shows that a feasible version of the proposed eHL test can detect slight miscalibrations in practically relevant sample…

On Testing Equal Conditional Predictive Ability Under Measurement Error

Yannick Hoga, Timo Dimitriadis

econ.EM · · arXiv ↗ · more like this ·

Loss functions are widely used to compare several competing forecasts. However, forecast comparisons are often based on mismeasured proxy variables for the true target. We introduce the concept of exact robustness to measurement error for loss functions and fully characterize this class of loss functions as the Bregman class. For such exactly robust loss functions, forecast loss differences are on average unaffected by the use of proxy variables and, thus, inference on conditional predictive ability can be carried out as usual. Moreover, we show that more precise proxies give predictive ability tests higher power in discriminating between competing forecasts. Simulations illustrate the different behavior of exactly robust and non-robust loss functions. An empirical application to US GDP growth rates demonstrates that it is easier to discriminate between forecasts issued at different…

Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary

Timo Dimitriadis, Xiaochun Liu, Julie Schnaitmann

econ.EM · · arXiv ↗ · more like this ·

We propose forecast encompassing tests for the Expected Shortfall (ES) jointly with the Value at Risk (VaR) based on flexible link (or combination) functions. Our setup allows testing encompassing for convex forecast combinations and for link functions which preclude crossings of the combined VaR and ES forecasts. As the tests based on these link functions involve parameters which are on the boundary of the parameter space under the null hypothesis, we derive and base our tests on nonstandard asymptotic theory on the boundary. Our simulation study shows that the encompassing tests based on our new link functions outperform tests based on unrestricted linear link functions for one-step and multi-step forecasts. We further illustrate the potential of the proposed tests in a real data analysis for forecasting VaR and ES of the S&P 500 index.

Testing Forecast Rationality for Measures of Central Tendency

Timo Dimitriadis, Andrew J. Patton, Patrick W. Schmidt

econ.EM · · arXiv ↗ · more like this ·

Rational respondents to economic surveys may report as a point forecast any measure of the central tendency of their (possibly latent) predictive distribution, for example the mean, median, mode, or any convex combination thereof. We propose tests of forecast rationality when the measure of central tendency used by the respondent is unknown. We overcome an identification problem that arises when the measures of central tendency are equal or in a local neighborhood of each other, as is the case for (exactly or nearly) symmetric distributions. As a building block, we also present novel tests for the rationality of mode forecasts. We apply our tests to income forecasts from the Federal Reserve Bank of New York's Survey of Consumer Expectations. We find these forecasts are rationalizable as mode forecasts, but not as mean or median forecasts. We also find heterogeneity in the measure of…

Forecast Encompassing Tests for the Expected Shortfall

Timo Dimitriadis, Julie Schnaitmann

q-fin.RM · · arXiv ↗ · more like this ·

We introduce new forecast encompassing tests for the risk measure Expected Shortfall (ES). The ES currently receives much attention through its introduction into the Basel III Accords, which stipulate its use as the primary market risk measure for the international banking regulation. We utilize joint loss functions for the pair ES and Value at Risk to set up three ES encompassing test variants. The tests are built on misspecification robust asymptotic theory and we investigate the finite sample properties of the tests in an extensive simulation study. We use the encompassing tests to illustrate the potential of forecast combination methods for different financial assets.

Regression Based Expected Shortfall Backtesting

Sebastian Bayer, Timo Dimitriadis

q-fin.RM · · arXiv ↗ · more like this ·

This paper introduces novel backtests for the risk measure Expected Shortfall (ES) following the testing idea of Mincer and Zarnowitz (1969). Estimating a regression framework for the ES stand-alone is infeasible, and thus, our tests are based on a joint regression for the Value at Risk and the ES, which allows for different test specifications. These ES backtests are the first which solely backtest the ES in the sense that they only require ES forecasts as input parameters. As the tests are potentially subject to model misspecification, we provide asymptotic theory under misspecification for the underlying joint regression. We find that employing a misspecification robust covariance estimator substantially improves the tests' performance. We compare our backtests to existing approaches and find that our tests outperform the competitors throughout all considered simulations. In an…