arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Valentin Thomas

19 papers · 2017–2026

2017: 22018: 12019: 12020: 12021: 12022: 02023: 32024: 32025: 42026: 2
2017peak 4 in 20252026

Frequent co-authors

Papers

showing 1–19 of 19 RSS

On the interplay between noise and curvature and its effect on optimization and generalization

Valentin Thomas, Fabian Pedregosa, Bart van Merriënboer +3 more

cs.LG · · cited 75× · arXiv ↗ · more like this ·

The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ultimate goal is good generalization performance, we clarify how both curvature and noise are relevant to properly estimate the generalization gap. Realizing that the limitations of some existing works stems from a confusion between these matrices, we also clarify the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients.

Independently Controllable Factors

Valentin Thomas, Jules Pondard, Emmanuel Bengio +6 more

cs.LG · · cited 72× · arXiv ↗ · more like this ·

It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that some of the causal factors could be discovered if the learner is allowed to interact with its environment. The agent can experiment with different actions and observe their effects. More specifically, we hypothesize that some of these factors correspond to aspects of the environment which are independently controllable, i.e., that there exists a policy and a learnable feature for each such aspect of the environment, such that this policy can yield changes in that feature with minimal changes to other features that explain the statistical variations in the observed data.…

Disentangling the independently controllable factors of variation by interacting with the world

Valentin Thomas, Emmanuel Bengio, William Fedus +6 more

stat.ML · · cited 70× · arXiv ↗ · more like this ·

It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that some of the causal factors could be discovered if the learner is allowed to interact with its environment. The agent can experiment with different actions and observe their effects. More specifically, we hypothesize that some of these factors correspond to aspects of the environment which are independently controllable, i.e., that there exists a policy and a learnable feature for each such aspect of the environment, such that this policy can yield changes in that feature with minimal changes to other features that explain the statistical variations in the observed data.…

Independently Controllable Features

Emmanuel Bengio, Valentin Thomas, Joelle Pineau +2 more

cs.LG · · cited 66× · arXiv ↗ · more like this ·

Finding features that disentangle the different causes of variation in real data is a difficult task, that has nonetheless received considerable attention in static domains like natural images. Interactive environments, in which an agent can deliberately take actions, offer an opportunity to tackle this task better, because the agent can experiment with different actions and observe their effects. We introduce the idea that in interactive environments, latent factors that control the variation in observed data can be identified by figuring out what the agent can control. We propose a naive method to find factors that explain or measure the effect of the actions of a learner, and test it in illustrative experiments.

TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction

Rasa Hosseinzadeh, Alex Labach, Zexin Xue +3 more

cs.LG · · cited 9× · arXiv ↗ · more like this ·

Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity. However, recent approaches with either cell-based architectures or retrieval have sacrificed efficiency for raw performance, restricting their utility in situations where compute is limited or inference speed is crucial. We adopt an alternate approach, sticking with row-based attention while incorporating long context pre-training to eliminate the need for retrieval. By combining this with architectural improvements and SSL pre-training on a newly-sourced, larger corpus of real data results, we present TabDPT-Turbo, a model that provides comparable default performance to TabDPT v1.1 on TabArena-Lite, CC18, and CTR23, at orders of magnitude faster. In our experiments, TabDPT-Turbo is the fastest model overall among leading foundation models. We have released the new model as TabDPT…

Bridging the Gap Between Target Networks and Functional Regularization

Alexandre Piché, Valentin Thomas, Rafael Pardinas +4 more

stat.ML · · cited 7× · arXiv ↗ · more like this ·

Bootstrapping is behind much of the successes of deep Reinforcement Learning. However, learning the value function via bootstrapping often leads to unstable training due to fast-changing target values. Target Networks are employed to stabilize training by using an additional set of lagging parameters to estimate the target values. Despite the popularity of Target Networks, their effect on the optimization is still misunderstood. In this work, we show that they act as an implicit regularizer which can be beneficial in some cases, but also have disadvantages such as being inflexible and can result in instabilities, even when vanilla TD(0) converges. To overcome these issues, we propose an explicit Functional Regularization alternative that is flexible and a convex regularizer in function space and we theoretically study its convergence. We conduct an experimental study across a range of…

Admission Control for URLLC Traffic with Computation Requirements in 5G and Beyond

Fidan Mehmeti, Valentin Thomas Haider, Wolfgang Kellerer

Computer Networks and Communications · mediaTUM (Technical University of Munich) · · cited 5× · DOI ↗ · more like this ·

One of the three types of services supported by 5G networks are Ultra-Reliable Low-Latency Communications, which are characterized by the stringent requirement to deliver packets within a very short time with a high reliability. Besides being successfully transmitted/received, these data need to be processed as well. To satisfy these strict requirements, one needs to determine both the required data rate and the processing rate, given the channel conditions and traffic intensity of the service. Moreover, with constraints on both the Radio Access Network and edge computing resources as well as with the competition between an ever-increasing number of users in cellular networks, a very important question which arises is that of admission control. This guarantees users will not suffer from deteriorating performance. In this paper, using analytical modeling, we derive admission control…

Joint $\alpha$-Fair Allocation of RAN and Computing Resources to URLLC Users in 5G

Valentin Thomas Haider, Fidan Mehmeti, Ana Cantarero +1 more

Computer Networks and Communications · IEEE Transactions on Vehicular Technology · · cited 4× · DOI ↗ · more like this ·

5G networks have emerged as the only viable solution to accomplish a satisfying performance level for various types of services, where each of them has very challenging traffic requirements. One of those services are ultra-reliable low-latency communications (URLLC), which are characterized by the stringent demand to deliver packets within a very short time with high reliability. Besides being successfully transmitted, the data must be processed as well. To maximize the number of users the network can serve, the interplay of the different resources must be understood and adequate resource allocation schemes must be devised. In this paper, we consider the joint allocation of uplink and downlink radio access network (RAN) and edge computing resources such that the traffic requirements of individual users are met and the network utility is maximized for different types of fairness. To this…

Understanding the Surprising Generalization Properties of Tabular Foundation Models

Nour Shaheen, Junwei Ma, Alex Labach +3 more

cs.LG · · cited 3× · arXiv ↗ · more like this ·

Tabular Foundation Models (TFMs) increasingly rely on in-context learning, where a model receives labelled examples at inference time and predicts labels for new inputs without updating its weights. Existing TFMs are typically trained on either massive synthetic corpora or very large collections of real datasets. In contrast, we show that surprisingly strong transfer can emerge from self-supervised pre-training on just a single real table. In this setting, we also find that tables tend to be either broadly useful or broadly poor regardless of downstream prediction task, and that the strongest predictor of usefulness is the number of features rather than the number of instances. This leads to a task-centric interpretation of tabular pre-training: the number and the quality of tasks are essential for the pre-training of TFMs. We show that the same task-centric perspective can help corpus…

Retrieval & Fine-Tuning for In-Context Tabular Models

Valentin Thomas, Junwei Ma, Rasa Hosseinzadeh +4 more

cs.LG · · cited 3× · arXiv ↗ · more like this ·

Tabular data is a pervasive modality spanning a wide range of domains, and the inherent diversity poses a considerable challenge for deep learning. Recent advancements using transformer-based in-context learning have shown promise on smaller and less complex datasets, but have struggled to scale to larger and more complex ones. To address this limitation, we propose a combination of retrieval and fine-tuning: we can adapt the transformer to a local subset of the data by collecting nearest neighbours, and then perform task-specific fine-tuning with this retrieved set of neighbours in context. Using TabPFN as the base model -- currently the best tabular in-context learner -- and applying our retrieval and fine-tuning scheme on top results in what we call a locally-calibrated PFN, or LoCalPFN. We conduct extensive evaluation on 95 datasets curated by TabZilla from OpenML, upon which we…

TabDPT: Scaling Tabular Foundation Models on Real Data

Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh +7 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

Tabular data is one of the most ubiquitous sources of information worldwide, spanning a wide variety of domains. This inherent heterogeneity has slowed the development of Tabular Foundation Models (TFMs) capable of fast generalization to unseen datasets. In-Context Learning (ICL) has recently emerged as a promising solution for TFMs, enabling dynamic adaptation to new tasks without additional tuning. While many studies have attempted to re-purpose large language models for tabular ICL, they have had limited success, so recent works have focused on developing tabular-specific foundation models. In this work, we propose an approach to combine ICL-based retrieval with self supervised learning to train tabular foundation models. We also investigate the utility of real vs. synthetic data for model pre-training, and show that real data can contain useful signal not easily captured in…

The Role of Baselines in Policy Gradient Optimization

Jincheng Mei, Wesley Chung, Valentin Thomas +3 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

We study the effect of baselines in on-policy stochastic policy gradient optimization, and close the gap between the theory and practice of policy optimization methods. Our first contribution is to show that the \emph{state value} baseline allows on-policy stochastic \emph{natural} policy gradient (NPG) to converge to a globally optimal policy at an $O(1/t)$ rate, which was not previously known. The analysis relies on two novel findings: the expected progress of the NPG update satisfies a stochastic version of the non-uniform Łojasiewicz (NŁ) inequality, and with probability 1 the state value baseline prevents the optimal action's probability from vanishing, thus ensuring sufficient exploration. Importantly, these results provide a new understanding of the role of baselines in stochastic policy gradient: by showing that the variance of natural policy gradient estimates remains unbounded…

Beyond variance reduction: Understanding the true impact of baselines on policy optimization

Wesley Chung, Valentin Thomas, Marlos C. Machado +1 more

cs.LG · · cited 1× · arXiv ↗ · more like this ·

Bandit and reinforcement learning (RL) problems can often be framed as optimization problems where the goal is to maximize average performance while having access only to stochastic estimates of the true gradient. Traditionally, stochastic optimization theory predicts that learning dynamics are governed by the curvature of the loss function and the noise of the gradient estimates. In this paper we demonstrate that this is not the case for bandit and RL problems. To allow our analysis to be interpreted in light of multi-step MDPs, we focus on techniques derived from stochastic optimization principles (e.g., natural policy gradient and EXP3) and we show that some standard assumptions from optimization theory are violated in these problems. We present theoretical results showing that, at least for bandit problems, curvature and noise are not sufficient to explain the learning dynamics and…

Deadly Hazard Without Risk? How Toxic Exposure Has Been Downplayed in Transnational Conflicts Over Carcinogen Assessment

Valentin Thomas

Sociology and Political Science · Comparativ · · cited 0× · DOI ↗ · more like this ·

In contemporary carcinogen identification systems, a key distinction is made between “hazard”– the inherent potential of an exposure to cause cancer – and “risk” – the probability of cancerarising from that exposure. Drawing on an unpublished set of archives from the InternationalAgency for Research on Cancer, this article traces the origins of this distinction. Contrary to theprevailing institutional narrative, the distinction emerges not from a straightforward scientificor regulatory consensus, but from a complex jurisdictional struggle among expert groups, eachseeking to define and prioritize different methods of assessing carcinogens. Bringing togethera range of actors from scientific, economic, and regulatory spheres, this distinction has enabledsome stakeholders to challenge expert assessments and delay political decision-making.

CausalPFN: Amortized Causal Effect Estimation via In-Context Learning

Vahid Balazadeh, Hamidreza Kamkari, Valentin Thomas +4 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Causal effect estimation from observational data is fundamental across various applications. However, selecting an appropriate estimator from dozens of specialized methods demands substantial manual effort and domain expertise. We present CausalPFN, a single transformer that amortizes this workflow: trained once on a large library of simulated data-generating processes that satisfy ignorability, it infers causal effects for new observational datasets out of the box. CausalPFN combines ideas from Bayesian causal inference with the large-scale training protocol of prior-fitted networks (PFNs), learning to map raw observations directly to causal effects without any task-specific adjustment. Our approach achieves superior average performance on heterogeneous and average treatment effect estimation benchmarks (IHDP, Lalonde, ACIC). Moreover, it shows competitive performance for real-world…

A Novel Sparse Transmission Scheme for Efficient Mobile Radio Communications

Caroline Zoll, Wolfgang Zirwas, Brenda Vilas Boas +3 more

Computer Networks and Communications · mediaTUM (Technical University of Munich) · · cited 0× · DOI ↗ · more like this ·

Applications like multi user and distributed MIMO can benefit from a sparse transmission scheme due to a reduced inter-cell and inter-cluster interference. In this paper, the so-called start stop bit method, which presents a new type of transmission with sparse resource usage is studied. Several enhancements are analyzed such as i) transmitting multiple stop bit sequences, ii) adding sparse stop bit areas representing artificial fractional time frequency shifts, iii) adding a given MCS over the stop bit areas, and iv) compressing the permutation matrix needed for multiple stop bit sequences. Bit error rate curves of the proposed transmission scheme are analyzed and suitable parameter sets are provided that lead to the highest throughput per set of resource elements for a given sparsity level in comparison with a conventional OFDMA system.

Toward Dynamic Frequency Planning for Reliable Connectivity in Mobile 6G in-X Subnetworks

Haider, Valentin Thomas; Mehmeti, Fidan; Kellerer, Wolfgang

Artificial Intelligence · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · cited 0× · OpenAlex ↗ · more like this ·

Within the research for the sixth generation of mobile networks, so-called in-X subnetworks (SNs) were proposed to support services with extreme performance requirements in geographically confined areas. To facilitate efficient spectrum usage and reliable connectivity within in-X SNs, adequate dynamic frequency planning methods for managing interference must be developed. This work provides background on in-X SNs and reviews related work on frequency planning and interference management. The problem of minimizing SN reconfigurations is identified as a key objective, while it is not addressed in the literature to date. Thus, the trade-off between minimizing subband usage and reducing SN reconfigurations is analyzed using both an already existing basic and a newly proposed advanced interference model. Preliminary results highlight how interference modeling and knowledge of future…

Generalization Can Emerge in Tabular Foundation Models From a Single Table

Junwei Ma, Nour Shaheen, Alex Labach +4 more

cs.LG · · arXiv ↗ · more like this ·

Deep tabular modelling increasingly relies on in-context learning where, during inference, a model receives a set of $(x,y)$ pairs as context and predicts labels for new inputs without weight updates. We challenge the prevailing view that broad generalization here requires pre-training on large synthetic corpora (e.g., TabPFN priors) or a large collection of real data (e.g., TabDPT training datasets), discovering that a relatively small amount of data suffices for generalization. We find that simple self-supervised pre-training on just a \emph{single} real table can produce surprisingly strong transfer across heterogeneous benchmarks. By systematically pre-training and evaluating on many diverse datasets, we analyze what aspects of the data are most important for building a Tabular Foundation Model (TFM) generalizing across domains. We then connect this to the pre-training procedure…

In-Context Data Distillation with TabPFN

Junwei Ma, Valentin Thomas, Guangwei Yu +1 more

cs.LG · · arXiv ↗ · more like this ·

Foundation models have revolutionized tasks in computer vision and natural language processing. However, in the realm of tabular data, tree-based models like XGBoost continue to dominate. TabPFN, a transformer model tailored for tabular data, mirrors recent foundation models in its exceptional in-context learning capability, being competitive with XGBoost's performance without the need for task-specific training or hyperparameter tuning. Despite its promise, TabPFN's applicability is hindered by its data size constraint, limiting its use in real-world scenarios. To address this, we present in-context data distillation (ICD), a novel methodology that effectively eliminates these constraints by optimizing TabPFN's context. ICD efficiently enables TabPFN to handle significantly larger datasets with a fixed memory budget, improving TabPFN's quadratic memory complexity but at the cost of a…