arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Jake Snell

19 papers · 2015–2025

2015: 12016: 02017: 12018: 22019: 02020: 22021: 12022: 12023: 52024: 32025: 3
2015peak 5 in 20232025

Frequent co-authors

Papers

showing 1–19 of 19 RSS

Prototypical Networks for Few-shot Learning

Jake Snell, Kevin Swersky, Richard S. Zemel

cs.LG · · cited 10,656× · arXiv ↗ · more like this ·

We propose prototypical networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each new class. Prototypical networks learn a metric space in which classification can be performed by computing distances to prototype representations of each class. Compared to recent approaches for few-shot learning, they reflect a simpler inductive bias that is beneficial in this limited-data regime, and achieve excellent results. We provide an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning. We further extend prototypical networks to zero-shot learning and achieve state-of-the-art results on the CU-Birds dataset.

Meta-Learning for Semi-Supervised Few-Shot Classification

Mengye Ren, Eleni Triantafillou, Sachin Ravi +5 more

cs.LG · · cited 1,458× · arXiv ↗ · more like this ·

In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for a learning algorithm is defined and trained on episodes representing different classification problems, each with a small labeled training set and its corresponding test set. In this work, we advance this few-shot classification paradigm towards a scenario where unlabeled examples are also available within each episode. We consider two situations: one where all unlabeled examples are assumed to belong to the same set of classes as the labeled examples of the episode, as well as the more challenging situation where examples from other distractor classes are also provided. To address this paradigm, we propose novel extensions of Prototypical Networks…

Learning to Generate Images with Perceptual Similarity Metrics

Jake Snell, Karl Ridgeway, Renjie Liao +3 more

cs.LG · · cited 209× · arXiv ↗ · more like this ·

Deep networks are increasingly being applied to problems involving image synthesis, e.g., generating images from textual descriptions and reconstructing an input image from a compact representation. Supervised training of image-synthesis networks typically uses a pixel-wise loss (PL) to indicate the mismatch between a generated image and its corresponding target image. We propose instead to use a loss function that is better calibrated to human perceptual judgments of image quality: the multiscale structural-similarity score (MS-SSIM). Because MS-SSIM is differentiable, it is easily incorporated into gradient-descent learning. We compare the consequences of using MS-SSIM versus PL loss on training deterministic and stochastic autoencoders. For three different architectures, we collected human judgments of the quality of image reconstructions. Observers reliably prefer images synthesized…

Learning Latent Subspaces in Variational Autoencoders

Jack Klys, Jake Snell, Richard Zemel

cs.LG · · cited 158× · arXiv ↗ · more like this ·

Variational autoencoders (VAEs) are widely used deep generative models capable of learning unsupervised latent representations of data. Such representations are often difficult to interpret or control. We consider the problem of unsupervised learning of features correlated to specific labels in a dataset. We propose a VAE-based generative model which we show is capable of extracting features correlated to binary labels in the data and structuring it in a latent subspace which is easy to interpret. Our model, the Conditional Subspace VAE (CSVAE), uses mutual information minimization to learn a low-dimensional latent subspace associated with each label that can easily be inspected and independently manipulated. We demonstrate the utility of the learned representations for attribute manipulation tasks on both the Toronto Face and CelebA datasets.

Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes

Jake Snell, Richard Zemel

cs.LG · · cited 8× · arXiv ↗ · more like this ·

Few-shot classification (FSC), the task of adapting a classifier to unseen classes given a small labeled dataset, is an important step on the path toward human-like machine learning. Bayesian methods are well-suited to tackling the fundamental issue of overfitting in the few-shot scenario because they allow practitioners to specify prior beliefs and update those beliefs in light of observed data. Contemporary approaches to Bayesian few-shot classification maintain a posterior distribution over model parameters, which is slow and requires storage that scales with model size. Instead, we propose a Gaussian process classifier based on a novel combination of Pólya-Gamma augmentation and the one-vs-each softmax approximation that allows us to efficiently marginalize over functions rather than model parameters. We demonstrate improved accuracy and uncertainty quantification on both standard…

ParaVerse: Co-design of a Parachute Rehearsal and Training Virtual-reality Enhanced Simulator for the Australian Defence Force: Combining a Generative Co-design Framework and an Agile Approach to Development

Brennen Mills, Martin Mašek, Julie Boston +6 more

Human-Computer Interaction · Research Square · · cited 3× · DOI ↗ · more like this ·

<title>Abstract</title> While co-design methods are crucial for developing digital educational interventions that are user-centred, contextually relevant, inclusive, and effective in meeting the diverse needs of learners and educators, little attention has been paid to the potential value of co-design processes for digital application development in the Defence context. This research gauged the efficacy of combining a generative co-design framework making use of agile and iterative co-design principles in an applied research and development project. The project produced an immersive virtual reality based digital solution in collaboration with the Australian Defence Force Special Operations Command (SOCOMD) Army. Specifically, the ParaVerse project sought to develop a solution considering the advanced tactics, techniques and procedures relevant to special operations soldiers for advanced…

Binary climate data visuals amplify perceived impact of climate change

Grace Liu, Jake Snell, Thomas L. Griffiths +1 more

Sociology and Political Science · · cited 2× · DOI ↗ · more like this ·

For much of the global population, climate change appears as a slow, gradual shift in daily weather. This leads many to perceive its impacts as minor and results in apathy (the "boiling frog" effect). How can we convey the urgency of the crisis when its impacts appear so subtle? Here, through a series of large-scale cognitive experiments (N=799), we find that presenting people with binary climate data (e.g., lake freeze history) significantly increases the perceived impact of climate change (Cohen's d = 0.40 [0.26, 0.54]) compared to continuous data (e.g., mean temperature). Computational modeling and follow-up experiments (N=398) suggest that binary data enhances perceived impact by creating an "illusion" of sudden shifts. Crucially, our approach does not involve selective data presentation but rather compares different datasets that reflect equivalent trends in climate change over…

Probing Few-Shot Generalization with Attributes

Mengye Ren, Eleni Triantafillou, Kuan-Chieh Wang +5 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

Despite impressive progress in deep learning, generalizing far beyond the training distribution is an important open challenge. In this work, we consider few-shot classification, and aim to shed light on what makes some novel classes easier to learn than others, and what types of learned representations generalize better. To this end, we define a new paradigm in terms of attributes -- simple building blocks of which concepts are formed -- as a means of quantifying the degree of relatedness of different concepts. Our empirical analysis reveals that supervised learning generalizes poorly to new attributes, but a combination of self-supervised pretraining with supervised finetuning leads to stronger generalization. The benefit of self-supervised pretraining and supervised finetuning is further investigated through controlled experiments using random splits of the attribute space, and we…

Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Models

Thomas P. Zollo, Todd Morrill, Zhun Deng +3 more

cs.LG · · cited 1× · arXiv ↗ · more like this ·

The recent explosion in the capabilities of large language models has led to a wave of interest in how best to prompt a model to perform a given task. While it may be tempting to simply choose a prompt based on average performance on a validation set, this can lead to a deployment where unexpectedly poor responses are generated, especially for the worst-off users. To mitigate this prospect, we propose Prompt Risk Control, a lightweight framework for selecting a prompt based on rigorous upper bounds on families of informative risk measures. We offer methods for producing bounds on a diverse set of metrics, including quantities that measure worst-case responses and disparities in generation quality across the population of users. In addition, we extend the underlying statistical bounding techniques to accommodate the possibility of distribution shifts in deployment. Experiments on…

Quantile Risk Control: A Flexible Framework for Bounding the Probability of High-Loss Predictions

Jake C. Snell, Thomas P. Zollo, Zhun Deng +2 more

cs.LG · · cited 1× · arXiv ↗ · more like this ·

Rigorous guarantees about the performance of predictive algorithms are necessary in order to ensure their responsible use. Previous work has largely focused on bounding the expected loss of a predictor, but this is not sufficient in many risk-sensitive applications where the distribution of errors is important. In this work, we propose a flexible framework to produce a family of bounds on quantiles of the loss distribution incurred by a predictor. Our method takes advantage of the order statistics of the observed loss values rather than relying on the sample mean alone. We show that a quantile is an informative way of quantifying predictive performance, and that our framework applies to a variety of quantile-based metrics, each targeting important subsets of the data distribution. We analyze the theoretical properties of our proposed method and demonstrate its ability to rigorously…

Meta-Learning at Scale for Large Language Models via Low-Rank Amortized Bayesian Meta-Learning

Liyi Zhang, Jake Snell, Thomas L. Griffiths

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Fine-tuning large language models (LLMs) with low-rank adaptation (LoRA) is a cost-effective way to incorporate information from a specific dataset. However, when a problem requires incorporating information from multiple datasets - as in few shot learning - generalization across datasets can be limited, driving up training costs. As a consequence, other approaches such as in-context learning are typically used in this setting. To address this challenge, we introduce an efficient method for adapting the weights of LLMs to multiple distributions, Amortized Bayesian Meta-Learning for LoRA (ABMLL). This method builds on amortized Bayesian meta-learning for smaller models, adapting this approach to LLMs by reframing where local and global variables are defined in LoRA and using a new hyperparameter to balance reconstruction accuracy and the fidelity of task-specific parameters to the global…

Conformal Prediction as Bayesian Quadrature

Jake C. Snell, Thomas L. Griffiths

cs.LG · · cited 0× · arXiv ↗ · more like this ·

As machine learning-based prediction systems are increasingly used in high-stakes situations, it is important to understand how such predictive models will perform upon deployment. Distribution-free uncertainty quantification techniques such as conformal prediction provide guarantees about the loss black-box models will incur even when the details of the models are hidden. However, such methods are based on frequentist probability, which unduly limits their applicability. We revisit the central aspects of conformal prediction from a Bayesian perspective and thereby illuminate the shortcomings of frequentist guarantees. We propose a practical alternative based on Bayesian quadrature that provides interpretable guarantees and offers a richer representation of the likely range of losses to be observed at test time.

Learning Human-Aligned Representations with Contrastive Learning and Generative Similarity

Raja Marjieh, Sreejan Kumar, Declan Campbell +4 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Humans rely on effective representations to learn from few examples and abstract useful information from sensory data. Inducing such representations in machine learning models has been shown to improve their performance on various benchmarks such as few-shot learning and robustness. However, finding effective training procedures to achieve that goal can be challenging as psychologically rich training data such as human similarity judgments are expensive to scale, and Bayesian models of human inductive biases are often intractable for complex, realistic domains. Here, we address this challenge by leveraging a Bayesian notion of generative similarity whereby two data points are considered similar if they are likely to have been sampled from the same distribution. This measure can be applied to complex generative processes, including probabilistic programs. We incorporate generative…

A Metalearned Neural Circuit for Nonparametric Bayesian Inference

Jake C. Snell, Gianluca Bencomo, Thomas L. Griffiths

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Most applications of machine learning to classification assume a closed set of balanced classes. This is at odds with the real world, where class occurrence statistics often follow a long-tailed power-law distribution and it is unlikely that all classes are seen in a single sample. Nonparametric Bayesian models naturally capture this phenomenon, but have significant practical barriers to widespread adoption, namely implementation complexity and computational inefficiency. To address this, we present a method for extracting the inductive bias from a nonparametric Bayesian model and transferring it to an artificial neural network. By simulating data with a nonparametric Bayesian prior, we can metalearn a sequence model that performs inference over an unlimited set of classes. After training, this "neural circuit" has distilled the corresponding inductive bias and can successfully perform…

Implicit Maximum a Posteriori Filtering via Adaptive Optimization

Gianluca M. Bencomo, Jake C. Snell, Thomas L. Griffiths

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Bayesian filtering approximates the true underlying behavior of a time-varying system by inverting an explicit generative model to convert noisy measurements into state estimates. This process typically requires either storage, inversion, and multiplication of large matrices or Monte Carlo estimation, neither of which are practical in high-dimensional state spaces such as the weight spaces of artificial neural networks. Here, we frame the standard Bayesian filtering problem as optimization over a time-varying objective. Instead of maintaining matrices for the filtering equations or simulating particles, we specify an optimizer that defines the Bayesian filter implicitly. In the linear-Gaussian setting, we show that every Kalman filter has an equivalent formulation using K steps of gradient descent. In the nonlinear setting, our experiments demonstrate that our framework results in…

Distribution-Free Statistical Dispersion Control for Societal Applications

Zhun Deng, Thomas P. Zollo, Jake C. Snell +2 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning. Previous work has focused mainly on bounding either the expected loss of a predictor or the probability that an individual prediction will incur a loss value in a specified range. However, for many high-stakes applications, it is crucial to understand and control the dispersion of a loss distribution, or the extent to which different members of a population experience unequal effects of algorithmic decisions. We initiate the study of distribution-free control of statistical dispersion measures with societal implications and propose a simple yet flexible framework that allows us to handle a much richer class of statistical functionals beyond previous work. Our methods are verified through experiments in toxic comment detection, medical imaging, and film…

Im-Promptu: In-Context Composition from Image Prompts

Bhishma Dedhia, Michael Chang, Jake C. Snell +2 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Large language models are few-shot learners that can solve diverse tasks from a handful of demonstrations. This implicit understanding of tasks suggests that the attention mechanisms over word tokens may play a role in analogical reasoning. In this work, we investigate whether analogical reasoning can enable in-context composition over composable elements of visual stimuli. First, we introduce a suite of three benchmarks to test the generalization properties of a visual in-context learner. We formalize the notion of an analogy-based in-context learner and use it to design a meta-learning framework called Im-Promptu. Whereas the requisite token granularity for language is well established, the appropriate compositional granularity for enabling in-context generalization in visual stimuli is usually unspecified. To this end, we use Im-Promptu to train multiple agents with different levels…

Exploring representation learning for flexible few-shot tasks

Mengye Ren, Eleni Triantafillou, Kuan-Chieh Wang +5 more

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

Existing approaches to few-shot learning deal with tasks that have persistent, rigid notions of classes. Typically, the learner observes data only from a fixed number of classes at training time and is asked to generalize to a new set of classes at test time. Two examples from the same class would always be assigned the same labels in any episode. In this work, we consider a realistic setting where the relationship between examples can change from episode to episode depending on the task context, which is not given to the learner. We define two new benchmark datasets for this flexible few-shot scenario, where the tasks are based on images of faces (Celeb-A) and shoes (Zappos50K). While classification baselines learn representations that work well for standard few-shot learning, they suffer in our flexible tasks since the classification criteria shift from training to testing. On the…

Improving Predictor Reliability with Selective Recalibration

Thomas P. Zollo, Zhun Deng, Jake C. Snell +2 more

cs.LG · · arXiv ↗ · more like this ·

A reliable deep learning system should be able to accurately express its confidence with respect to its predictions, a quality known as calibration. One of the most effective ways to produce reliable confidence estimates with a pre-trained model is by applying a post-hoc recalibration method. Popular recalibration methods like temperature scaling are typically fit on a small amount of data and work in the model's output space, as opposed to the more expressive feature embedding space, and thus usually have only one or a handful of parameters. However, the target distribution to which they are applied is often complex and difficult to fit well with such a function. To this end we propose \textit{selective recalibration}, where a selection model learns to reject some user-chosen proportion of the data in order to allow the recalibrator to focus on regions of the input space that can be…