arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Robin Chan

26 papers · 2002–2026

2002: 12003: 02004: 02005: 02006: 02007: 02008: 02009: 02010: 12011: 02012: 02013: 02014: 02015: 02016: 02017: 02018: 12019: 42020: 32021: 12022: 42023: 32024: 52025: 12026: 2
2002peak 5 in 20242026

Frequent co-authors

Papers

showing 1–25 of 26 RSS

Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation

Robin Chan, Matthias Rottmann, Hanno Gottschalk

cs.CV · · cited 225× · arXiv ↗ · more like this ·

Deep neural networks (DNNs) for the semantic segmentation of images are usually trained to operate on a predefined closed set of object classes. This is in contrast to the "open world" setting where DNNs are envisioned to be deployed to. From a functional safety point of view, the ability to detect so-called "out-of-distribution" (OoD) samples, i.e., objects outside of a DNN's semantic space, is crucial for many applications such as automated driving. A natural baseline approach to OoD detection is to threshold on the pixel-wise softmax entropy. We present a two-step procedure that significantly improves that approach. Firstly, we utilize samples from the COCO dataset as OoD proxy and introduce a second training objective to maximize the softmax entropy on these samples. Starting from pretrained semantic segmentation networks we re-train a number of DNNs on different in-distribution…

SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

Robin Chan, Krzysztof Lis, Svenja Uhlemeyer +6 more

cs.CV · · cited 208× · arXiv ↗ · more like this ·

State-of-the-art semantic or instance segmentation deep neural networks (DNNs) are usually trained on a closed set of semantic classes. As such, they are ill-equipped to handle previously-unseen objects. However, detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving, especially if they appear on the road ahead. While some methods have tackled the tasks of anomalous or out-of-distribution object segmentation, progress remains slow, in large part due to the lack of solid benchmarks; existing datasets either consist of synthetic data, or suffer from label inconsistencies. In this paper, we bridge this gap by introducing the "SegmentMeIfYouCan" benchmark. Our benchmark addresses two tasks: Anomalous object segmentation, which considers any previously-unseen object category; and road obstacle segmentation, which focuses on…

Application of Decision Rules for Handling Class Imbalance in Semantic Segmentation

Robin Chan, Matthias Rottmann, Fabian Hüger +2 more

cs.CV · · cited 42× · arXiv ↗ · more like this ·

As part of autonomous car driving systems, semantic segmentation is an essential component to obtain a full understanding of the car's environment. One difficulty, that occurs while training neural networks for this purpose, is class imbalance of training data. Consequently, a neural network trained on unbalanced data in combination with maximum a-posteriori classification may easily ignore classes that are rare in terms of their frequency in the dataset. However, these classes are often of highest interest. We approach such potential misclassifications by weighting the posterior class probabilities with the prior class probabilities which in our case are the inverse frequencies of the corresponding classes in the training dataset. More precisely, we adopt a localized method by computing the priors pixel-wise such that the impact can be analyzed at pixel level as well. In our…

Working memory for complex figures: An fMRI comparison of letter and fractal n-back tasks.

J. Daniel Ragland, Bruce I. Turetsky, Ruben C. Gur +5 more

Artificial Intelligence · Neuropsychology · · cited 32× · DOI ↗ · more like this ·

n -back letter and fractal tasks were administered to 11 participants during functional magnetic resonance imaging to test process specificity theories of prefrontal cortex (PFC) function and assess task validity.Tasks were matched on accuracy, but fractal n-back responses were slower and more conservative.Maintenance (1-back minus 0-back) activated inferior parietal and dorsolateral PFC, with additional activation in right ventrolateral PFC during letter n-back and left lingual gyrus during fractal n-back.Maintenance plus manipulation (2-back minus 0-back) activated inferior parietal, Broca's area, insula, and dorsolateral and ventral PFC, with greater right dorsolateral PFC activation for letter n-back.Manipulation only (2-back minus 1-back) produced additional and equivalent dorsolateral PFC and anterior cingulate activation in both tasks.Results support fractal n-back validity and…

Controlled False Negative Reduction of Minority Classes in Semantic Segmentation

Robin Chan, Matthias Rottmann, Fabian Hüger +2 more

Artificial Intelligence · PUB – Publications at Bielefeld University (Bielefeld University) · · cited 12× · DOI ↗ · more like this ·

In semantic segmentation datasets, classes of high importance are oftentimes underrepresented, e.g., humans in street scenes. Neural networks are usually trained to reduce the overall number of errors, attaching identical loss to errors of all kinds. However, this is not necessarily aligned with human intuition. For instance, an overlooked pedestrian seems more severe than an incorrectly detected one. One possible remedy is to deploy different decision rules by introducing class priors that assign more weight to underrepresented classes. While reducing the false negatives of the underrepresented class, at the same time this leads to a considerable increase of false positive indications. In this work, we combine decision rules with methods for false positive detection. Therefore, we fuse false negative detection with uncertainty based false positive meta classification. We present the…

Application of Maximum Likelihood Decision Rules for Handling Class Imbalance in Semantic Segmentation

Robin Chan, Matthias Rottmann, Hanno Gottschalk +2 more

Artificial Intelligence · Proceedings of the 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conference · · cited 12× · DOI ↗ · more like this ·

Chan RK-W, Rottmann M, Gottschalk H, Hüger F, Schlicht P. Application of Maximum Likelihood Decision Rules for Handling Class Imbalance in Semantic Segmentation. In: Baraldi P, Maio FD, Zio E, eds. <em>Proceedings of the 30th European Safety and Reliability Conference and 15th Probabilistic Safety Assessment and Management Conference</em>. Singapore: Research Publishing Services; 2020: 3065-3072.

The Ethical Dilemma when (not) Setting up Cost-based Decision Rules in Semantic Segmentation

Robin Chan, Matthias Rottmann, Radin Dardashti +3 more

cs.CV · · cited 12× · arXiv ↗ · more like this ·

Neural networks for semantic segmentation can be seen as statistical models that provide for each pixel of one image a probability distribution on predefined classes. The predicted class is then usually obtained by the maximum a-posteriori probability (MAP) which is known as Bayes rule in decision theory. From decision theory we also know that the Bayes rule is optimal regarding the simple symmetric cost function. Therefore, it weights each type of confusion between two different classes equally, e.g., given images of urban street scenes there is no distinction in the cost function if the network confuses a person with a street or a building with a tree. Intuitively, there might be confusions of classes that are more important to avoid than others. In this work, we want to raise awareness of the possibility of explicitly defining confusion costs and the associated ethical difficulties…

Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes

Youssef Shoeb, Robin Chan, Gesina Schwalbe +3 more

cs.CV · · cited 8× · arXiv ↗ · more like this ·

In the life cycle of highly automated systems operating in an open and dynamic environment, the ability to adjust to emerging challenges is crucial. For systems integrating data-driven AI-based components, rapid responses to deployment issues require fast access to related data for testing and reconfiguration. In the context of automated driving, this especially applies to road obstacles that were not included in the training data, commonly referred to as out-of-distribution (OoD) road obstacles. Given the availability of large uncurated recordings of driving scenes, a pragmatic approach is to query a database to retrieve similar scenarios featuring the same safety concerns due to OoD road obstacles. In this work, we extend beyond identifying OoD road obstacles in video streams and offer a comprehensive approach to extract sequences of OoD road obstacles using text queries, thereby…

MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation

Robin Chan, Matthias Rottmann, Fabian Hüger +2 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

In semantic segmentation datasets, classes of high importance are oftentimes underrepresented, e.g., humans in street scenes. Neural networks are usually trained to reduce the overall number of errors, attaching identical loss to errors of all kinds. However, this is not necessarily aligned with human intuition. For instance, an overlooked pedestrian seems more severe than an incorrectly detected one. One possible remedy is to deploy different decision rules by introducing class priors which assigns larger weight to underrepresented classes. While reducing the false-negatives of the underrepresented class, at the same time this leads to a considerable increase of false-positive indications. In this work, we combine decision rules with methods for false-positive detection. We therefore fuse false-negative detection with uncertainty based false-positive meta classification. We present…

An Estelle-C compiler for automatic protocol implementation

Robin I Chan

Artificial Intelligence · cIRcle (University of British Columbia) · · cited 6× · DOI ↗ · more like this ·

Over the past few years, much experience has been gained in semi-automatic protocol implementation using an existing Estelle-C compiler developed at the University of British Columbia. However, with the continual evolution of the Estelle language, that compiler is now obsolete. The present study found substantial syntactic and semantic differences between the Estelle language as implemented by the existing compiler and that specified in the latest ISO document to warrant the construction of a new Estelle-C compiler. The result is a new compiler which translates Estelle as defined in the second version of the ISO Draft Proposal 9074 into the programming language C. The new Estelle-C compiler addresses issues such as dynamic reconfiguration of modules and maintenance of priority relationships among nested modules. A run-time environment capable of supporting the new Estelle features is…

Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities

Matthias Rottmann, Pascal Colling, Thomas-Paul Hack +4 more

cs.CV · · cited 5× · arXiv ↗ · more like this ·

We present a method that "meta" classifies whether seg-ments predicted by a semantic segmentation neural networkintersect with the ground truth. For this purpose, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input scene's size. We aggregate these dispersion measures segment-wise and derive metrics that are well-correlated with the segment-wise IoU of prediction and ground truth. This procedure yields an almost plug and play post-processing tool to rate the prediction quality of semantic segmentation networks on segment level. This is especially relevant for monitoring neural networks in online applications like automated driving or medical imaging where reliability is of utmost importance. In our tests, we use publicly available state-of-the-art networks trained on the Cityscapes…

What should AI see? Using the Public's Opinion to Determine the Perception of an AI

Robin Chan, Radin Dardashti, Meike Osinski +7 more

cs.LG · · cited 4× · arXiv ↗ · more like this ·

Deep neural networks (DNN) have made impressive progress in the interpretation of image data, so that it is conceivable and to some degree realistic to use them in safety critical applications like automated driving. From an ethical standpoint, the AI algorithm should take into account the vulnerability of objects or subjects on the street that ranges from "not at all", e.g. the road itself, to "high vulnerability" of pedestrians. One way to take this into account is to define the cost of confusion of one semantic category with another and use cost-based decision rules for the interpretation of probabilities, which are the output of DNNs. However, it is an open problem how to define the cost structure, who should be in charge to do that, and thereby define what AI-algorithms will actually "see". As one possible answer, we follow a participatory approach and set up an online survey to…

Detecting and Learning the Unknown in Semantic Segmentation

Robin Chan, Svenja Uhlemeyer, Matthias Rottmann +1 more

cs.CV · · cited 4× · arXiv ↗ · more like this ·

Semantic segmentation is a crucial component for perception in automated driving. Deep neural networks (DNNs) are commonly used for this task and they are usually trained on a closed set of object classes appearing in a closed operational domain. However, this is in contrast to the open world assumption in automated driving that DNNs are deployed to. Therefore, DNNs necessarily face data that they have never encountered previously, also known as anomalies, which are extremely safety-critical to properly cope with. In this work, we first give an overview about anomalies from an information-theoretic perspective. Next, we review research in detecting semantically unknown objects in semantic segmentation. We demonstrate that training for high entropy responses on anomalous objects outperforms other recent methods, which is in line with our theoretical findings. Moreover, we examine a…

Detection of False Positive and False Negative Samples in Semantic Segmentation

Matthias Rottmann, Kira Maag, Robin Chan +3 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

In recent years, deep learning methods have outperformed other methods in image recognition. This has fostered imagination of potential application of deep learning technology including safety relevant applications like the interpretation of medical images or autonomous driving. The passage from assistance of a human decision maker to ever more automated systems however increases the need to properly handle the failure modes of deep learning modules. In this contribution, we review a set of techniques for the self-monitoring of machine-learning algorithms based on uncertainty quantification. In particular, we apply this to the task of semantic segmentation, where the machine learning algorithm decomposes an image according to semantic categories. We discuss false positive and false negative error modes at instance-level and review techniques for the detection of such errors that have…

FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation

Riza Velioglu, Robin Chan, Barbara Hammer

cs.CV · · cited 1× · arXiv ↗ · more like this ·

In the realm of fashion object detection and segmentation for online shopping images, existing state-of-the-art fashion parsing models encounter limitations, particularly when exposed to non-model-worn apparel and close-up shots. To address these failures, we introduce FashionFail; a new fashion dataset with e-commerce images for object detection and segmentation. The dataset is efficiently curated using our novel annotation tool that leverages recent foundation models. The primary objective of FashionFail is to serve as a test bed for evaluating the robustness of models. Our analysis reveals the shortcomings of leading models, such as Attribute-Mask R-CNN and Fashionformer. Additionally, we propose a baseline approach using naive data augmentation to mitigate common failure cases and improve model robustness. Through this work, we aim to inspire and support further research in fashion…

Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals

Robin Chan, Afra Amini, Mennatallah El-Assady

cs.CL · · cited 1× · arXiv ↗ · more like this ·

We present a human-in-the-loop dashboard tailored to diagnosing potential spurious features that NLI models rely on for predictions. The dashboard enables users to generate diverse and challenging examples by drawing inspiration from GPT-3 suggestions. Additionally, users can receive feedback from a trained NLI model on how challenging the newly created example is and make refinements based on the feedback. Through our investigation, we discover several categories of spurious correlations that impact the reasoning of NLI models, which we group into three categories: Semantic Relevance, Logical Fallacies, and Bias. Based on our findings, we identify and describe various research opportunities, including diversifying training data and assessing NLI models' robustness by creating adversarial test suites.

LU-Net: Invertible Neural Networks Based on Matrix Factorization

Robin Chan, Sarina Penquitt, Hanno Gottschalk

cs.LG · · cited 1× · arXiv ↗ · more like this ·

LU-Net is a simple and fast architecture for invertible neural networks (INN) that is based on the factorization of quadratic weight matrices $\mathsf{A=LU}$, where $\mathsf{L}$ is a lower triangular matrix with ones on the diagonal and $\mathsf{U}$ an upper triangular matrix. Instead of learning a fully occupied matrix $\mathsf{A}$, we learn $\mathsf{L}$ and $\mathsf{U}$ separately. If combined with an invertible activation function, such layers can easily be inverted whenever the diagonal entries of $\mathsf{U}$ are different from zero. Also, the computation of the determinant of the Jacobian matrix of such layers is cheap. Consequently, the LU architecture allows for cheap computation of the likelihood via the change of variables formula and can be trained according to the maximum likelihood principle. In our numerical experiments, we test the LU-net architecture as generative model…

MGT: Extending Virtual Try-Off to Multi-Garment Scenarios

Riza Velioglu, Petra Bevandic, Robin Chan +1 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Computer vision is transforming fashion industry through Virtual Try-On (VTON) and Virtual Try-Off (VTOFF). VTON generates images of a person in a specified garment using a target photo and a standardized garment image, while a more challenging variant, Person-to-Person Virtual Try-On (p2p-VTON), uses a photo of another person wearing the garment. VTOFF, in contrast, extracts standardized garment images from photos of clothed individuals. We introduce Multi-Garment TryOffDiff (MGT), a diffusion-based VTOFF model capable of handling diverse garment types, including upper-body, lower-body, and dresses. MGT builds on a latent diffusion architecture with SigLIP-based image conditioning to capture garment characteristics such as shape, texture, and pattern. To address garment diversity, MGT incorporates class-specific embeddings, achieving state-of-the-art VTOFF results on VITON-HD and…

New advances in universal approximation with neural networks of minimal width

Dennis Rochau, Robin Chan, Hanno Gottschalk

cs.NE · · cited 0× · arXiv ↗ · more like this ·

We prove several universal approximation results at minimal or near-minimal width for approximation of $L^p(\mathbb{R}^{d_x}, \mathbb{R}^{d_y})$ and $C^0(\mathbb{R}^{d_x}, \mathbb{R}^{d_y})$ on compact sets. Our approach uses a unified coding scheme that yields explicit constructions relying only on standard analytic tools. We show that feedforward neural networks with two leaky ReLU activations $σ_α$, $σ_{-α}$ achieve the optimal width $\max\{d_x, d_y\}$ for $L^p$ approximation, while a single leaky ReLU $σ_α$ achieves width $\max\{2, d_x, d_y\}$, providing an alternative proof of the results of Cai et al. (2023). By generalizing to stepped leaky ReLU activations, we extend these results to uniform approximation of continuous functions while identifying sets of activation functions compatible with gradient-based training. Since our constructions pass through an intermediate dimension…

What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages

Nadav Borenstein, Anej Svete, Robin Chan +5 more

cs.CL · · cited 0× · arXiv ↗ · more like this ·

What can large language models learn? By definition, language models (LM) are distributions over strings. Therefore, an intuitive way of addressing the above question is to formalize it as a matter of learnability of classes of distributions over strings. While prior work in this direction focused on assessing the theoretical limits, in contrast, we seek to understand the empirical learnability. Unlike prior empirical work, we evaluate neural LMs on their home turf-learning probabilistic languages-rather than as classifiers of formal languages. In particular, we investigate the learnability of regular LMs (RLMs) by RNN and Transformer LMs. We empirically test the learnability of RLMs as a function of various complexity parameters of the RLM and the hidden state size of the neural LM. We find that the RLM rank, which corresponds to the size of linear space spanned by the logits of its…

Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects

Kira Maag, Robin Chan, Svenja Uhlemeyer +2 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

In this work we present two video test data sets for the novel computer vision (CV) task of out of distribution tracking (OOD tracking). Here, OOD objects are understood as objects with a semantic class outside the semantic space of an underlying image segmentation algorithm, or an instance within the semantic space which however looks decisively different from the instances contained in the training data. OOD objects occurring on video sequences should be detected on single frames as early as possible and tracked over their time of appearance as long as possible. During the time of appearance, they should be segmented as precisely as possible. We present the SOS data set containing 20 video sequences of street scenes and more than 1000 labeled frames with up to two OOD objects. We furthermore publish the synthetic CARLA-WildLife data set that consists of 26 video sequences containing…

Uncertainty Quantification and Resource-Demanding Computer Vision Applications of Deep Learning

Julian Burghoff, Robin Chan, Hanno Gottschalk +4 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Bringing deep neural networks (DNNs) into safety critical applications such as automated driving, medical imaging and finance, requires a thorough treatment of the model's uncertainties. Training deep neural networks is already resource demanding and so is also their uncertainty quantification. In this overview article, we survey methods that we developed to teach DNNs to be uncertain when they encounter new object classes. Additionally, we present training methods to learn from only a few labels with help of uncertainty quantification. Note that this is typically paid with a massive overhead in computation of an order of magnitude and more compared to ordinary network training. Finally, we survey our work on neural architecture search which is also an order of magnitude more resource demanding then ordinary network training.

Detecting Pose Estimation Failures via Keypoint Self-Consistency

Robin Chan

cs.CV · · arXiv ↗ · more like this ·

One common approach to pose estimation involves predicting object keypoints in an image, followed by using Perspective-n-Point algorithms to compute the object's rotation and translation relative to the camera. While rotations preserve object shapes, this property is often neglected in keypoint-based pose estimation methods, where keypoints are typically predicted independently from each other. As imprecise keypoint predictions negatively affects pose estimation accuracy, it also limits its reliability in downstream tasks. In this work, we explore whether such inaccurate pose estimates can be identified by simply examining spatial locations between 2D keypoints. We propose a set of hand-crafted geometric features that capture the self-consistency of keypoint predictions, including pairwise distances, reprojection consistency, as well as render and mask consistency. Despite its…

Impact of the Einstein Telescope's duty cycle on the estimation of binary black holes parameters

Luca Negri, Thomas C. K. Ng, Thibeau Wouters +13 more

gr-qc · · arXiv ↗ · more like this ·

The geometry of the Einstein Telescope, the proposed next-generation European gravitational-wave observatory, is yet to be finalized. Two competing designs are under consideration: a nested triangular configuration (ET-Δ) and two separated L-shaped detectors (ET-2L). Extensive prior comparisons of ET designs established the scientific landscape using the Fisher-information-matrix formalism and identified that duty-cycle-induced single-detector operation is precisely the regime where this approximation becomes less reliable, underscoring the need for a refined, principled treatment of the duty cycle. In this manuscript, we build on that foundation by revisiting the comparison with full Bayesian parameter estimation of gravitational-wave signals from binary black-hole mergers, projected onto a simulated Einstein Telescope that incorporates a refined duty cycle modelled via continuous-time…