Predictive stability modeling is widely used in pharmaceutical development to establish product shelf-life by identifying the kinetic parameters that govern degradation behavior and using these parameters to extrapolate out in time. Recent approaches typically assume linear degradation dynamics and rely on the Arrhenius equation to model temperature-specific reaction rates. However, observed degradation profiles may reflect nonlinear reaction dynamics and more complex temperature-specific reaction rates, potentially leading to reduced accuracy in long-term predictions and motivating a more flexible approach. We propose a novel Bayesian framework which extends predictive stability modeling beyond simple Arrhenius low-order kinetic dynamics and accommodates complex degradation mechanisms. The degradation process is modeled through the Sestak-Berggren kinetic formulation, a flexible…
Grid-scale battery energy storage systems (BESS) are becoming increasingly important in short-term energy markets, where they help balance fluctuations in variable renewable generation. In many countries, the continuous intraday market is the most important wholesale market for BESS to monetize their flexibility. In this paper, we propose an extension to the myopic rolling intrinsic policy that incorporates intraday price forecasts tailored specifically to the BESS trading problem. While a large literature addresses intraday price forecasting, most existing work focuses on volume-weighted prices during the last hours of trading and consequently fails to capture the features most relevant to BESS trading. Our approach addresses this gap by introducing a novel target variable derived from the limit order book and its dynamics. The resulting forecasts are designed to inform the…
The increasing availability of detailed football performance data has created new opportunities for identifying player archetypes and informing recruitment and tactical decision-making. Most existing clustering approaches assign each player to a single group, despite the reality that many modern footballers perform hybrid roles spanning multiple stylistic categories. Performance data for 314 outfield players from the 2024/25 English Premier League season are analysed using a Gaussian mixture model (GMM) applied to principal component representations of player statistics. Unlike hard-assignment methods such as k-means, the GMM provides probabilistic cluster memberships, allowing uncertainty and role hybridity to be quantified. Seven player archetypes are identified in the full dataset. Posterior membership probabilities reveal a subset of players whose profiles span multiple clusters…
In many applications, variability itself carries important predictive information beyond the mean, and its association with an outcome may further depend on the underlying mean trajectory. However, existing functional regression approaches primarily rely on mean-based representations of functional predictors, while variability is often ignored or reduced to scalar summaries. We propose a functional regression framework that incorporates latent mean and variance processes as predictors, together with their mean-variance interaction, allowing the predictive contribution of variability to depend on the underlying mean trajectory. We establish consistency and convergence rates for the estimated coefficient functions and interaction surface, accounting for reconstruction errors, increasing observation frequency, and diverging truncation dimensions without requiring independence between the…
After the 2021-2022 energy crisis, day-ahead prices in Nordic bidding zones no longer move together. We ask whether this divergence also changes the information carried by prices. The answer matters for integrating variable renewable energy (VRE), because it can help identify where wind and solar forecasts are worth producing, publishing, and using. We measured the co-movement of each Norwegian and Swedish zone with Germany-Luxembourg (DE-LU) in 2018-2025. We then used a single model, selected on 2024 data, to forecast 2025 prices at 11:00 on the previous day and tested which inputs improved accuracy in each zone. The zones moved alike until 2019, but co-movement diverged in Sweden in 2020 and in Norway in 2021, and the split has persisted. In 2024, four southern zones co-moved strongly with DE-LU (intraday correlations of 0.54-0.71), while four northern zones co-moved weakly…
Artificial intelligence (AI) is quickly expanding the capacity of organizations to produce and analyze data. We argue that this expansion shifts the leverage of data scientists from performing individual analyses to assessing the credibility and importance of findings and determining when evidence generalizes across contexts. Drawing on a roundtable of academic researchers and industry practitioners, we examine three ways in which data scientists who work on causal inference and experimentation can help organizations turn more analytical capacity into better decisions: preserving and accumulating the learning value of experimentation beyond individual tests; choosing and combining experimental, observational, and synthetic evidence; and ensuring evaluation keeps pace with AI as its capabilities and uses evolve. In each area, we distinguish established practice from open questions. We…
Artificial intelligence has the potential to transform how public services are delivered, with important implications for public service productivity. But how widely is the technology actually being used - and is it making any difference? Our survey of 3,362 public sector workers across the UK, the USA, South Africa and the EU finds widespread use of AI in every region. Our findings suggest that this spread is being driven by bottom-up individual adoption rather than top-down institutional adoption. In the EU, for example, 65\% of respondents use AI, while only 19\% say their organisation has formally adopted it. Most users are positive about its effects, with 79\% saying AI has improved their productivity. However, only 29\% say it has increased what is expected of them at work, raising questions about how truly transformative the technology is so far. Across regions, South Africa is a…
New Zealand's Clean Car Discount (CCD) was a feebate scheme that offered rebates for low-emission vehicles from July 2021, added fees for high-emission vehicles from April 2022, and was abolished at the end of December 2023. Shortly afterwards, road user charges were introduced for electric vehicles (EVs). We study how EV adoption responded to the introduction and the abrupt repeal of the scheme, using monthly vehicle registration data from 2014 to 2024. A structural break analysis following Bai and Perron, combined with a regression discontinuity in time, shows that breaks in the registrations of battery electric, plug-in hybrid and conventional vehicles coincide with the policy dates. To estimate causal effects, we apply a difference-in-differences approach with Queensland, Australia, as a comparison region and Newey-West standard errors. Because the New Zealand market was about four…
Cancer drugs are often evaluated for efficacy in mice before proceeding to human clinical trials. To do this, human tumors are engrafted onto immune-incompetent mice to create a patient-derived xenograft (PDX). Engrafted tumors can be propagated to evaluate multiple drugs, but the time and costs involved in PDX studies limit high throughput drug screens in mice. Here we present GrAFT-PDX, a Bayesian model of PDX drug screens that aims to reduce the PDX screening burden by enabling imputation of missing experiments and early stopping of likely unsuccessful drugs. Unlike conventional survival or growth models, GrAFT-PDX jointly predicts tumor growth and survival outcomes using a learned latent function decomposition. In experiments on a large-scale PDX dataset, GrAFT-PDX outperforms Cox proportional hazards models at predicting time of death, outperforms machine learning baselines at…
Unequal recovery from pandemic learning losses raises concerns about widening achievement gaps between economically disadvantaged students and their peers. This study seeks to identify districts whose mathematics gaps widened in every annual interval from 2022 to 2025. The analysis uses estimates from the Stanford Education Data Archive (SEDA) for 5,600 U.S. school districts. Identifying such districts requires accounting for correlated errors and differing precision in the estimated annual changes, as well as multiplicity. The true annual gap changes in each district form an effect vector, and persistent widening means that every component is positive. To address this class of multiple testing problems, we propose an empirical Bayes mirror procedure. It uses an estimated distribution of effect vectors to guide ranking and covariance-adjusted mirror observations to calibrate selection.…
Synthetic data augmentation has been proposed as a way to artificially supplement clinical datasets to address fairness concerns in downstream classification algorithms. This approach is closely related to upweighting or upsampling methods employed for the problem of class imbalance, which have shown mixed empirical results in practice. Synthetic data augmentation also runs the risk of overfitting to the noise present in subgroups with relatively small sample sizes, raising both generalizability and transportability concerns. We present an empirical evaluation of synthetic data augmentation for fairness in tabular data using simulation studies as well as a real-world application of in-hospital mortality classification for heart failure patients in two distinct data samples. We implement upsampling and upweighting methods as benchmarks for the implicit fairness objective of synthetic…
Survey simulations require realistic combinations of answers, but predictive accuracy alone does not establish that a model can generate them. We evaluate the option probabilities returned by Jev, a TypeSafe decision model, using linked responses to 20 binary items from five archived survey waves. We compare population-only prediction, prediction given recorded answers, and sequential completion that uses generated answers to predict later responses. Joint comparisons hold the starting population information fixed, and two three-question cases are completed in both question orders. We evaluate predictive scores, answer frequencies, and associations against the survey records. Recorded answers can improve predictive scores while associations remain distorted. Large conditional errors also occur. For example, among men who described themselves as having traditional values, the model…
Comparing soccer teams that rarely or never meet is a fundamental challenge in sports analytics: clubs play mostly within their national leagues, and national teams within their confederations. We propose a rating framework that places clubs and national teams worldwide on a common probabilistic scale. Building on a recent extension of Glicko with order effects and strength-dependent draw probabilities, we adapt the model to soccer with goal-scoring adjustments, a decomposition of home advantage, extra time, league-aware rating initialization and rules of promotion and relegation that do not inflate the levels of leagues; national teams start from the levels of their confederations. The models are evaluated on about 1.2 million club matches of 252 leagues from 166 countries (2000-2026) and 12.6 thousand national team matches (1998-2026). On held-out seasons, the modified Glicko model is…
Factorial and Reduced K-means estimate a partition and a low-dimensional clustering subspace jointly, which makes them attractive when the variables that separate groups are not those that dominate total variance. On contaminated high-dimensional data they face three linked problems: trimming can make the scatter matrices singular, the criterion has many local optima, and the rank, number of groups and trimming proportion are unknown and interact. We propose REG-HD-TFKM and REG-HD-TRKM, regularized trimmed versions of the two methods, together with a selection procedure that handles these problems as separate decisions. A ridge-stabilized spectral step keeps every fit well defined; heterogeneous multistart search explores alternative solutions; method-specific agreement and reproducibility scores choose among them; and trimming is applied only when a radial diagnostic or the path over…
Extreme weather events are critical for climate impact assessments. While physical models can simulate unprecedented extremes, for example, through ensemble boosting, this approach is computationally demanding, often restricting output to coarse spatial resolutions. Probabilistic generative downscaling offers a solution by enabling efficient sampling from the entire climate distribution, including its extreme tails. Here, we first demonstrate that machine learning generated regional climate model heatwaves over Paris are physically realistic. We then leverage an emulated machine learning dataset to perform analog-based downscaling of coarse global climate model ensemble boosting events. Our hybrid framework effectively bridges high-resolution regional detail with rare-event sampling, providing an efficient tool for extreme event downscaling and capturing multi-model uncertainty.
We propose a nested ensemble learning framework that utilizes Topological Data Analysis (TDA) to extract and integrate topological features from graph data, with the goal of improving performance on classification and regression tasks. Our approach computes persistence diagrams (PDs) using lower-star filtrations induced by three filter functions: closeness, betweenness, and degree2 centrality. To overcome the limitation of relying on a single filter, these PDs are integrated through a data-driven, three-level architecture. At Level-0, diverse base models are independently trained on the topological features extracted for each filter function. At Level-1, a meta-learner combines the predictions of these base models for each filter to form filter-specific ensembles. Finally, at Level-2, a meta-learner integrates the outputs of these filter-specific ensembles to produce the final…
Two recent studies on ship turnaround time at Brazilian ports report models with R-squared scores above 0.80 on data from the national waterway transportation agency. We audit their published datasets and reveal these results are inflated by structural data leakage. Specifically, 97 percent of berthing events in their random row-level split appear in both training and testing partitions. We demonstrate that a model's R-squared drops from 0.82 to 0.20 when the data is properly split at the berthing level. We further document additional methodological flaws, including survivorship bias, undocumented target truncation, and the retention of variables that algebraically reconstruct the target. To address these issues, we propose a reproducible pipeline that aggregates cargo to the berthing level, utilizes conditional imputation, and represents high-cardinality cargo variables through…
Sleep is scored afterwards in 30-second epochs as wake (W), the transitional stage N1, or N2. We study EEG forecasts, issued every 30 s from the preceding 30 s of EEG alone, of whether the first scored N2 epoch will begin within five minutes, on Sleep-EDF. A Brier score pooled over decisions made before and after the input is scored N1 does not show when the gain arose. In our own prespecified one-shot held-out comparisons, the spectral model SpecLR-18 improved on a training prevalence constant by 18.3% in the internal test, whereas its prespecified external result was inconclusive. Joining the retrospectively scored stage of each input epoch to the same frozen predictions, almost all of that internal-test Brier difference was attributed to N1-scored decisions; a diagnostic reference told only the scored stage reached 26.3% in the internal test and 29.5% in the external cohort, above…
Competing risks arise when individuals may experience multiple mutually exclusive event types. Competing events are frequently treated as censoring and analysed using Kaplan--Meier (KM) methods, which overestimate real-world event risks. While the existence of KM overestimation is well known, its determinants, magnitude, and inferential properties remain poorly characterised. We develop a nonparametric framework that treats competing risks overestimation as a statistical quantity in its own right. We show that competing event frequency alone is insufficient to characterise KM overestimation and develop an event ordering theory describing how temporal event allocation influences overestimation. Based on these results, we derive a closed-form Total Overestimation Diagnostic (TOD) that quantifies total KM overestimation directly from routine survival analysis output. We establish exact…
We study the parameter estimation for the stochastic logistic and Lotka--Volterra diffusions from discrete, irregularly spaced observations corrupted by measurement noise, a setting typical of microbiome time series. For the logistic equation, the growth rate acts as a time change and does not enter the stationary law, so it can be identified only from the temporal structure of the data. We prove that the maximum-likelihood estimator of the Euler scheme, with the observations taken as the states, coincides with the regression of log-increments on abundances used in the literature, and we characterise what this estimator cannot see under measurement noise: for the linearised dynamics, the law of consecutive pairs of observations is invariant along an explicit one-parameter family of relaxation rates and noise levels, and the regression converges to a limit set by the noise. Treating the…
Spatial blocking is a common technique used in designed agricultural experiments. Spatial blocking, however, requires important assumptions about the spatial dependence patterns in the field. Statistical models of blocked agricultural experiments often show spatial structure in the residuals, suggesting that the original model may benefit from additional assumptions regarding the spatial dependence patterns present in the field. We propose a Bayesian model for checking block membership and estimating spatial patterns missing in the original block design. Our model is particularly motivated by cases with mis-specified blocks. To accomplish this, we embed a regression tree within a Bayesian statistical model to estimate spatial effects with a block-like spatial structure. We illustrate the benefits of our model using simulated data mimicking four typical scenarios we have encountered in…
The stiffness and cracking behaviour of concrete are influenced by the interfacial transition zone (ITZ), the thin shell of paste that surrounds each aggregate. Yet the ITZ's mechanical properties are poorly characterized and are usually assigned to mesoscale models through ad hoc reduction factors. Because the ITZ is only tens of micrometres thick, its stiffness must be inferred indirectly from micromechanical tests, and the resulting estimates vary widely between studies and are rarely reported with a quantified uncertainty. We present a workflow that calibrates a homogenized ITZ model against the surface displacements of several micro-cantilever bending specimens and returns its stiffness descriptors as posterior distributions rather than single values. The calibration is posed as a Bayesian inverse problem with a model-discrepancy term, sampled with a surrogate-assisted Bouncy…
Estimates of the limits of human performance are usually built on record books, on the premise that a record is the best an athlete could do. In Olympic weightlifting the premise fails, because the competitor declares the weight before each of three attempts per lift and only the heaviest successful attempt is recorded, so a record is a lower bound, chosen by the athlete, on a maximal capability never observed. We treat the maximal lift of each competitor on each competition day as a latent variable, of which every attempt is a trial and every declared weight a measurement. Its distribution at a given bodyweight follows an extreme value mixture model with an estimated threshold, and an athlete-specific random effect links the competitions of each athlete. The model is fitted to the 117,892 attempts of 5,040 athletes in 141 international competitions between 2017 and 2026. The upper tail…
We study the problem of testing isotropy, i.e., distributional invariance under orthogonal transformations, for random fields taking values in abstract spaces. We propose a novel approach based on computing local directional characteristics, which are uniformly distributed under the assumption of isotropy. Our work is motivated by an application in material science, where we are testing the isotropy of function-valued porosity profiles of paper sheets, whose morphology is resolved by $μ$-CT imaging. While considerably more general, the presented methodology is competitive with existing isotropy tests for real-valued random fields.
A buyer who repeatedly participates in auctions for related items often faces a dynamic value problem: what they win today changes what tomorrow's items are worth. Despite the empirical importance of such effects, especially in display advertising, the algorithmic literature on repeated auctions has largely focused on the learning problem, by treating each opportunity as a contextual bandit, while sidestepping the planning problem induced by the dependence of current value on past wins. We introduce DynamAuction, a synthetic environment for single-buyer sequential bidding. The reward from each win depends on the buyer's history of previous wins, summarized through a carry-over state. The environment supports first- and second-price auctions, non-homogeneous Poisson arrivals, delayed feedback, saturation, and recency effects. We also introduce a new class of Model Predictive Control…