arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Aurelien Lucchi

108 papers · 2009–2026

2009: 12010: 22011: 12012: 12013: 22014: 12015: 32016: 52017: 92018: 112019: 102020: 92021: 82022: 112023: 52024: 72025: 122026: 10
2009peak 12 in 20252026

Frequent co-authors

Papers

showing 1–25 of 108 RSS

SLIC Superpixels Compared to State-of-the-Art Superpixel Methods

Radhakrishna Achanta, Anil Shaji, Kevin Smith +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 9,110× · DOI ↗ · more like this ·

Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.

Quantum Generative Adversarial Networks for Learning and Loading Random Distributions

Christa Zoufal, Aurélien Lucchi, Stefan Woerner

quant-ph · · cited 638× · arXiv ↗ · more like this ·

Quantum algorithms have the potential to outperform their classical counterparts in a variety of tasks. The realization of the advantage often requires the ability to load classical data efficiently into quantum states. However, the best known methods require $\mathcal{O}\left(2^n\right)$ gates to load an exact representation of a generic data structure into an $n$-qubit state. This scaling can easily predominate the complexity of a quantum algorithm and, thereby, impair potential quantum advantage. Our work presents a hybrid quantum-classical algorithm for efficient, approximate quantum state loading. More precisely, we use quantum Generative Adversarial Networks (qGANs) to facilitate efficient learning and loading of generic probability distributions -- implicitly given by data samples -- into quantum states. Through the interplay of a quantum channel, such as a variational quantum…

Stabilizing Training of Generative Adversarial Networks through Regularization

Kevin Roth, Aurelien Lucchi, Sebastian Nowozin +1 more

cs.LG · · cited 486× · arXiv ↗ · more like this ·

Deep generative models based on Generative Adversarial Networks (GANs) have demonstrated impressive sample quality but in order to work they require a careful choice of architecture, parameter initialization, and selection of hyper-parameters. This fragility is in part due to a dimensional mismatch or non-overlapping support between the model distribution and the data distribution, causing their density ratio and the associated f-divergence to be undefined. We overcome this fundamental limitation and propose a new regularization approach with low computational cost that yields a stable GAN training procedure. We demonstrate the effectiveness of this regularizer across several architectures trained on common benchmark image generation tasks. Our regularization turns GAN models into reliable building blocks for deep learning.

Escaping Saddles with Stochastic Gradients

Hadi Daneshmand, Jonas Kohler, Aurelien Lucchi +1 more

cs.LG · · cited 179× · arXiv ↗ · more like this ·

We analyze the variance of stochastic gradients along negative curvature directions in certain non-convex machine learning models and show that stochastic gradients exhibit a strong component along these directions. Furthermore, we show that - contrary to the case of isotropic noise - this variance is proportional to the magnitude of the corresponding eigenvalues and not decreasing in the dimensionality. Based upon this observation we propose a new assumption under which we show that the injection of explicit, isotropic noise usually applied to make gradient descent escape saddle points can successfully be replaced by a simple SGD step. Additionally - and under the same condition - we derive the first convergence rate for plain SGD to a second-order stationary point in a number of iterations that is independent of the problem dimension.

Variance Reduced Stochastic Gradient Descent with Neighbors

Thomas Hofmann, Aurelien Lucchi, Simon Lacoste-Julien +1 more

cs.LG · · cited 159× · arXiv ↗ · more like this ·

Stochastic Gradient Descent (SGD) is a workhorse in machine learning, yet its slow convergence can be a computational bottleneck. Variance reduction techniques such as SAG, SVRG and SAGA have been proposed to overcome this weakness, achieving linear convergence. However, these methods are either based on computations of full gradients at pivot points, or on keeping per data point corrections in memory. Therefore speed-ups relative to SGD may need a minimal number of epochs in order to materialize. This paper investigates algorithms that can exploit neighborhood structure in the training data to share and re-use information about past stochastic gradients across data points, which offers advantages in the transient optimization phase. As a side-product we provide a unified convergence analysis for a family of variance reduction algorithms, which we call memorization algorithms. We…

Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification

Jan Deriu, Aurelien Lucchi, Valeria De Luca +5 more

cs.CL · · cited 137× · arXiv ↗ · more like this ·

This paper presents a novel approach for multi-lingual sentiment classification in short texts. This is a challenging task as the amount of training data in languages other than English is very limited. Previously proposed multi-lingual approaches typically require to establish a correspondence to English for which powerful classifiers are already available. In contrast, our method does not require such supervision. We leverage large amounts of weakly-supervised data in various languages to train a multi-layer convolutional network and demonstrate the importance of using pre-training of such networks. We thoroughly evaluate our approach on various multi-lingual datasets, including the recent SemEval-2016 sentiment prediction benchmark (Task 4), where we achieved state-of-the-art performance. We also compare the performance of our model trained individually for each language to a variant…

Probabilistic Bag-Of-Hyperlinks Model for Entity Linking

Octavian-Eugen Ganea, Marina Ganea, Aurelien Lucchi +2 more

cs.CL · · cited 135× · arXiv ↗ · more like this ·

Many fundamental problems in natural language processing rely on determining what entities appear in a given text. Commonly referenced as entity linking, this step is a fundamental component of many NLP tasks such as text understanding, automatic summarization, semantic search or machine translation. Name ambiguity, word polysemy, context dependencies and a heavy-tailed distribution of entities contribute to the complexity of this problem. We here propose a probabilistic approach that makes use of an effective graphical model to perform collective entity disambiguation. Input mentions (i.e.,~linkable token spans) are disambiguated jointly across an entire document by combining a document-level prior of entity co-occurrences with local information captured from mentions and their surrounding context. The model is based on simple sufficient statistics extracted from data, thus relying on…

Variational Quantum Boltzmann Machines

Christa Zoufal, Aurélien Lucchi, Stefan Woerner

quant-ph · · cited 131× · arXiv ↗ · more like this ·

This work presents a novel realization approach to Quantum Boltzmann Machines (QBMs). The preparation of the required Gibbs states, as well as the evaluation of the loss function's analytic gradient is based on Variational Quantum Imaginary Time Evolution, a technique that is typically used for ground state computation. In contrast to existing methods, this implementation facilitates near-term compatible QBM training with gradients of the actual loss function for arbitrary parameterized Hamiltonians which do not necessarily have to be fully-visible but may also include hidden units. The variational Gibbs state approximation is demonstrated with numerical simulations and experiments run on real quantum hardware provided by IBM Quantum. Furthermore, we illustrate the application of this variational QBM approach to generative and discriminative learning tasks using numerical simulation.

Local Saddle Point Optimization: A Curvature Exploitation Approach

Leonard Adolphs, Hadi Daneshmand, Aurelien Lucchi +1 more

cs.LG · · cited 116× · arXiv ↗ · more like this ·

Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima. We propose a novel optimization approach that exploits curvature information in order to escape from these undesired stationary points. We prove that different optimization methods, including gradient method and Adagrad, equipped with curvature exploitation can escape non-optimal stationary points. We also provide empirical results on common saddle point problems which confirm the advantage of using curvature exploitation.

SwissCheese at SemEval-2016 Task 4: Sentiment Classification Using an Ensemble of Convolutional Neural Networks with Distant Supervision

Jan Deriu, Maurice Gonzenbach, Fatih Uzdilli +3 more

Artificial Intelligence · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 111× · DOI ↗ · more like this ·

In this paper, we propose a classifier for predicting message-level sentiments of English micro-blog messages from Twitter. Our method builds upon the convolutional sentence embedding approach proposed by (Severyn and Moschitti, 2015a; Severyn and Moschitti, 2015b). We leverage large amounts of data with distant supervision to train an ensemble of 2-layer convolutional neural networks whose predictions are combined using a random forest classifier. Our approach was evaluated on the datasets of the SemEval-2016 competition (Task 4) outperforming all other approaches for the Message Polarity Classification task.

Learning for Structured Prediction Using Approximate Subgradient Descent with Working Sets

Aurélien Lucchi, Yunpeng Li, Pascal Fua

Artificial Intelligence · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 110× · DOI ↗ · more like this ·

We propose a working set based approximate sub gradient descent algorithm to minimize the margin-sensitive hinge loss arising from the soft constraints in max-margin learning frameworks, such as the structured SVM. We focus on the setting of general graphical models, such as loopy MRFs and CRFs commonly used in image segmentation, where exact inference is intractable and the most violated constraints can only be approximated, voiding the optimality guarantees of the structured SVM's cutting plane algorithm as well as reducing the robustness of existing sub gradient based methods. We show that the proposed method obtains better approximate sub gradients through the use of working sets, leading to improved convergence properties and increased reliability. Furthermore, our method allows new constraints to be randomly sampled instead of computed using the more expensive approximate…

An Online Learning Approach to Generative Adversarial Networks

Paulina Grnarova, Kfir Y. Levy, Aurelien Lucchi +2 more

cs.LG · · cited 93× · arXiv ↗ · more like this ·

We consider the problem of training generative models with a Generative Adversarial Network (GAN). Although GANs can accurately model complex distributions, they are known to be difficult to train due to instabilities caused by a difficult minimax optimization problem. In this paper, we view the problem of training GANs as finding a mixed strategy in a zero-sum game. Building on ideas from online learning we propose a novel training method named Chekhov GAN 1 . On the theory side, we show that our method provably converges to an equilibrium for semi-shallow GAN architectures, i.e. architectures where the discriminator is a one layer network and the generator is arbitrary. On the practical side, we develop an efficient heuristic guided by our theoretical results, which we apply to commonly used deep GAN architectures. On several real world tasks our approach exhibits improved stability…

Are spatial and global constraints really necessary for segmentation?

Aurélien Lucchi, Yunpeng Li, Xavier Boix +2 more

Computer Vision and Pattern Recognition · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 72× · DOI ↗ · more like this ·

Many state-of-the-art segmentation algorithms rely on Markov or Conditional Random Field models designed to enforce spatial and global consistency constraints. This is often accomplished by introducing additional latent variables to the model, which can greatly increase its complexity. As a result, estimating the model parameters or computing the best maximum a posteriori (MAP) assignment becomes a computationally expensive task. In a series of experiments on the PASCAL and the MSRC datasets, we were unable to find evidence of a significant performance increase attributed to the introduction of such constraints. On the contrary, we found that similar levels of performance can be achieved using a much simpler design that essentially ignores these constraints. This more simple approach makes use of the same local and global features to leverage evidence from the image, but instead…

A Variance Reduced Stochastic Newton Method

Aurelien Lucchi, Brian McWilliams, Thomas Hofmann

cs.LG · · cited 52× · arXiv ↗ · more like this ·

Quasi-Newton methods are widely used in practise for convex loss minimization problems. These methods exhibit good empirical performance on a wide variety of tasks and enjoy super-linear convergence to the optimal solution. For large-scale learning problems, stochastic Quasi-Newton methods have been recently proposed. However, these typically only achieve sub-linear convergence rates and have not been shown to consistently perform well in practice since noisy Hessian approximations can exacerbate the effect of high-variance stochastic gradient estimates. In this work we propose Vite, a novel stochastic Quasi-Newton algorithm that uses an existing first-order technique to reduce this variance. Without exploiting the specific form of the approximate Hessian, we show that Vite reaches the optimum at a geometric rate with a constant step-size when dealing with smooth strongly convex…

TinkerSheets

Guillaume Zufferey, Patrick Jermann, Aurélien Lucchi +1 more

Human-Computer Interaction · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 51× · DOI ↗ · more like this ·

This paper describes TinkerSheets, a paper-based interface to tangible simulations. The proposed interface combines the advantages of form-based input and paper. Form-based input allows to set an arbitrary number of parameters. Using paper as a medium for the interface keeps the interaction modality consistently physical. TinkerSheets are also used as an output screen to display summarized information about the simulation. A user study conducted in an authentic context shows how the characteristics of the interface shape real world usage. We also describe how the affordances of this control and visualization interface support the co-design of interaction with end-users.

An empirical evaluation of touch and tangible interfaces for tabletop displays

Aurélien Lucchi, Patrick Jermann, Guillaume Zufferey +1 more

Human-Computer Interaction · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 48× · DOI ↗ · more like this ·

Tabletop systems have become quite popular in recent years, during which there was considerable enthusiasm for the development of new interfaces. In this paper, we establish a comparison between touch and tangible interfaces. We set up an experiment involving several actions like translation and rotation. We recruited 40 participants to take part in a user study and we present our results with a discussion on the design of touch and tangible interfaces. Our contribution is an empirical study showing that overall, the tangible interface is much faster but under certain conditions, the touch interface could gain the upper hand.

Learning Structured Models for Segmentation of 2-D and 3-D Imagery

Aurélien Lucchi, Pablo Márquez-Neila, Carlos Becker +4 more

Computer Vision and Pattern Recognition · IEEE Transactions on Medical Imaging · · cited 38× · DOI ↗ · more like this ·

Efficient and accurate segmentation of cellular structures in microscopic data is an essential task in medical imaging. Many state-of-the-art approaches to image segmentation use structured models whose parameters must be carefully chosen for optimal performance. A popular choice is to learn them using a large-margin framework and more specifically structured support vector machines (SSVM). Although SSVMs are appealing, they suffer from certain limitations. First, they are restricted in practice to linear kernels because the more powerful nonlinear kernels cause the learning to become prohibitively expensive. Second, they require iteratively finding the most violated constraints, which is often intractable for the loopy graphical models used in image segmentation. This requires approximation that can lead to reduced quality of learning. In this paper, we propose three novel techniques…

A Distributed Second-Order Algorithm You Can Trust

Celestine Dünner, Aurelien Lucchi, Matilde Gargiani +3 more

cs.LG · · cited 33× · arXiv ↗ · more like this ·

Due to the rapid growth of data and computational resources, distributed optimization has become an active research area in recent years. While first-order methods seem to dominate the field, second-order methods are nevertheless attractive as they potentially require fewer communication rounds to converge. However, there are significant drawbacks that impede their wide adoption, such as the computation and the communication of a large Hessian matrix. In this paper we present a new algorithm for distributed training of generalized linear models that only requires the computation of diagonal blocks of the Hessian matrix on the individual workers. To deal with this approximate information we propose an adaptive approach that - akin to trust-region methods - dynamically adapts the auxiliary model to compensate for modeling errors. We provide theoretical rates of convergence for a wide…

Adversarially Robust Training through Structured Gradient Regularization

Kevin Roth, Aurelien Lucchi, Sebastian Nowozin +1 more

stat.ML · · cited 24× · arXiv ↗ · more like this ·

We propose a novel data-dependent structured gradient regularizer to increase the robustness of neural networks vis-a-vis adversarial perturbations. Our regularizer can be derived as a controlled approximation from first principles, leveraging the fundamental link between training with noise and regularization. It adds very little computational overhead during learning and is simple to implement generically in standard deep learning frameworks. Our experiments provide strong evidence that structured gradient regularization can act as an effective first line of defense against attacks based on low-level signal corruption.

An Optimal Policy for Target Localization with Application to Electron Microscopy

Raphael Sznitman, Aurélien Lucchi, Peter I. Frazier +2 more

Artificial Intelligence · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 22× · OpenAlex ↗ · more like this ·

This paper considers the task of finding a target location by making a limited number of sequential observation. Each observation results from evaluating an imperfect classifier of a chosen cost and accuracy on an interval of chosen length and position. Within a Bayesian framework, we study the problem of minimizing an objective that combines the entropy of the posterior distribution with the cost of the questions asked. In this problem, we show that the one-step lookahead policy is Bayes-optimal for any arbitrary time horizon. Moreover, this one-step lookahead policy is easy to compute and implement. We then use this policy in the context of localizing mitochondria in electron microscope images, and experimentally show that significant speed ups in acquisition can be gained, while maintaining near equal image quality at target locations, when compared to current policies. Proceedings…

Momentum Improves Optimization on Riemannian Manifolds

Foivos Alimisis, Antonio Orvieto, Gary Bécigneul +1 more

math.OC · · cited 9× · arXiv ↗ · more like this ·

We develop a new Riemannian descent algorithm that relies on momentum to improve over existing first-order methods for geodesically convex optimization. In contrast, accelerated convergence rates proved in prior work have only been shown to hold for geodesically strongly-convex objective functions. We further extend our algorithm to geodesically weakly-quasi-convex objectives. Our proofs of convergence rely on a novel estimate sequence that illustrates the dependency of the convergence rate on the curvature of the manifold. We validate our theoretical results empirically on several optimization problems defined on the sphere and on the manifold of positive definite matrices.

A Sub-sampled Tensor Method for Non-convex Optimization

Aurelien Lucchi, Jonas Kohler

math.OC · · cited 9× · arXiv ↗ · more like this ·

We present a stochastic optimization method that uses a fourth-order regularized model to find local minima of smooth and potentially non-convex objective functions with a finite-sum structure. This algorithm uses sub-sampled derivatives instead of exact quantities. The proposed approach is shown to find an $(ε_1,ε_2,ε_3)$-third-order critical point in at most $\bigO\left(\max\left(ε_1^{-4/3}, ε_2^{-2}, ε_3^{-4}\right)\right)$ iterations, thereby matching the rate of deterministic approaches. In order to prove this result, we derive a novel tensor concentration inequality for sums of tensors of any order that makes explicit use of the finite-sum structure of the objective function.

A Semi-supervised Framework for Image Captioning

Wenhu Chen, Aurelien Lucchi, Thomas Hofmann

cs.CV · · cited 9× · arXiv ↗ · more like this ·

State-of-the-art approaches for image captioning require supervised training data consisting of captions with paired image data. These methods are typically unable to use unsupervised data such as textual data with no corresponding images, which is a much more abundant commodity. We here propose a novel way of using such textual data by artificially generating missing visual information. We evaluate this learning approach on a newly designed model that detects visual concepts present in an image and feed them to a reviewer-decoder architecture with an attention mechanism. Unlike previous approaches that encode visual concepts using word embeddings, we instead suggest using regional image features which capture more intrinsic information. The main benefit of this architecture is that it synthesizes meaningful thought vectors that capture salient image properties and then applies a soft…

Supervoxel-Based Segmentation of EM Image Stacks with Learned Shape Features

Aurélien Lucchi, Kevin Smith, Radhakrishna Achanta +2 more

Computer Vision and Pattern Recognition · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · · cited 9× · OpenAlex ↗ · more like this ·

Abstract—Immense amounts of high resolution data are now routinely produced thanks to recent advances in EM imaging. While a strong demand for automated analysis now exists, it is stifled by the lack of robust automatic 3D segmentation techniques. State-of-the-art Computer Vision algorithms designed to operate on natural 2D images tend to perform poorly when applied to EM image stacks for a number of reasons. The sheer size of a typical EM image stack renders many segmentation schemes intractable. Most approaches rely on local statistics that easily become confused when confronted with the noise and textures found within EM image stacks. The assumption that strong image gradients always correspond to object boundaries is violated by cluttered membranes belonging to numerous objects. In this work, we propose an automated graph partitioning scheme that addresses these issues. It reduces…

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

Sotiris Anagnostidis, Dario Pavllo, Luca Biggio +3 more

cs.CL · · cited 8× · arXiv ↗ · more like this ·

Autoregressive Transformers adopted in Large Language Models (LLMs) are hard to scale to long sequences. Despite several works trying to reduce their computational cost, most of LLMs still adopt attention layers between all pairs of tokens in the sequence, thus incurring a quadratic cost. In this study, we present a novel approach that dynamically prunes contextual information while preserving the model's expressiveness, resulting in reduced memory and computational requirements during inference. Our method employs a learnable mechanism that determines which uninformative tokens can be dropped from the context at any point across the generation process. By doing so, our approach not only addresses performance concerns but also enhances interpretability, providing valuable insight into the model's decision-making process. Our technique can be applied to existing pre-trained models…