arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Theo Gevers

195 papers · 1992–2026

1992: 11993: 01994: 21995: 01996: 01997: 01998: 11999: 12000: 22001: 12002: 52003: 32004: 72005: 72006: 52007: 92008: 62009: 82010: 142011: 62012: 62013: 82014: 122015: 52016: 22017: 62018: 92019: 42020: 62021: 52022: 92023: 122024: 162025: 122026: 5
1992peak 16 in 20242026

Frequent co-authors

Papers

showing 1–25 of 195 RSS

Segmentation as selective search for object recognition

Koen E. A. van de Sande, Jasper Uijlings, Theo Gevers +1 more

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 754× · DOI ↗ · more like this ·

For object recognition, the current state-of-the-art is based on exhaustive search. However, to enable the use of more expensive features and classifiers and thereby progress beyond the state-of-the-art, a selective search strategy is needed. Therefore, we adapt segmentation as a selective search by reconsidering segmentation: We propose to generate many approximate locations over few and precise object delineations because (1) an object whose location is never generated can not be recognised and (2) appearance and immediate nearby context are most effective for object recognition. Our method is class-independent and is shown to cover 96.7% of all objects in the Pascal VOC 2007 test set using only 1,536 locations per image. Our selective search enables the use of the more expensive bag-of-words method which we use to substantially improve the state-of-the-art by up to 8.5% for 8 out of…

PicToSeek: combining color and shape invariant features for image retrieval

Theo Gevers, A.W.M. Smeulders

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 485× · DOI ↗ · more like this ·

We aim at combining color and shape invariants for indexing and retrieving images. To this end, color models are proposed independent of the object geometry, object pose, and illumination. From these color models, color invariant edges are derived from which shape invariant features are computed. Computational methods are described to combine the color and shape invariants into a unified high-dimensional invariant feature set for discriminatory object retrieval. Experiments have been conducted on a database consisting of 500 images taken from multicolored man-made objects in real world scenes. From the theoretical and experimental results it is concluded that object retrieval based on composite color and shape invariant features provides excellent retrieval accuracy. Object retrieval based on color invariants provides very high retrieval accuracy whereas object retrieval based entirely…

Boosting color saliency in image feature detection

Joost van de Weijer, Theo Gevers, Andrew D. Bagdanov

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 293× · DOI ↗ · more like this ·

The aim of salient feature detection is to find distinctive local events in images. Salient features are generally determined from the local differential structure of images. They focus on the shape-saliency of the local neighborhood. The majority of these detectors are luminance-based, which has the disadvantage that the distinctiveness of the local color information is completely ignored in determining salient image features. To fully exploit the possibilities of salient point detection in color images, color distinctiveness should be taken into account in addition to shape distinctiveness. In this paper, color distinctiveness is explicitly incorporated into the design of saliency detection. The algorithm, called color saliency boosting, is based on an analysis of the statistics of color image derivatives. Color saliency boosting is designed as a generic method easily adaptable to…

Evaluation of color descriptors for object and scene recognition

Koen E. A. van de Sande, Theo Gevers, Cees G. M. Snoek

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 231× · DOI ↗ · more like this ·

Image category recognition is important to access visual information on the level of objects and scene types. So far, intensity-based descriptors have been widely used. To increase illumination invariance and discriminative power, color descriptors have been proposed only recently. As many descriptors exist, a structured overview of color invariant descriptors in the context of image category recognition is required.

Accurate eye center location and tracking using isophote curvature

Roberto Valenti, Theo Gevers

Human-Computer Interaction · UvA-DARE (University of Amsterdam) · · cited 210× · DOI ↗ · more like this ·

The ubiquitous application of eye tracking is precluded by the requirement of dedicated and expensive hardware, such as infrared high definition cameras. Therefore, systems based solely on appearance (i.e. not involving active infrared illumination) are being proposed in literature. However, although these systems are able to successfully locate eyes, their accuracy is significantly lower than commercial eye tracking devices. Our aim is to perform very accurate eye center location and tracking, using a simple Web cam. By means of a novel relevance mechanism, the proposed method makes use of isophote properties to gain invariance to linear lighting changes (contrast and brightness), to achieve rotational invariance and to keep low computational costs. In this paper we test our approach for accurate eye location and robustness to changes in illumination and pose, using the BioIDand the…

Image saliency by isocentric curvedness and color

Roberto Valenti, Nicu Sebe, Theo Gevers

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 158× · DOI ↗ · more like this ·

In this paper we propose a novel computational method to infer visual saliency in images. The method is based on the idea that salient objects should have local characteristics that are different than the rest of the scene, being edges, color or shape. By using a novel operator, these characteristics are combined to infer global information. The obtained information is used as a weighting for the output of a segmentation algorithm so that the salient object in the scene can easily be distinguished from the background. The proposed approach is fast and it does not require any learning. The experimentation shows that the system can enhance interesting objects in images and it is able to correctly locate the same object annotated by humans with an F-measure of 85.61% when the object size is known, and 79.19% when the object size is unknown, improving the state of the art performance on a…

Selection and Fusion of Color Models for Image Feature Detection

H.M.G. Stokman, Theo Gevers

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 120× · DOI ↗ · more like this ·

The choice of a color model is of great importance for many computer vision algorithms (e.g., feature detection, object recognition, and tracking) as the chosen color model induces the equivalence classes to the actual algorithms. As there are many color models available, the inherent difficulty is how to automatically select a single color model or, alternatively, a weighted subset of color models producing the best result for a particular task. The subsequent hurdle is how to obtain a proper fusion scheme for the algorithms so that the results are combined in an optimal setting. To achieve proper color model selection and fusion of feature detection algorithms, in this paper, we propose a method that exploits nonperfect correlation between color models or feature detection algorithms derived from the principles of diversification. As a consequence, a proper balance is obtained between…

Words Matter: Scene Text for Image Classification and Retrieval

Sezer Karaoğlu, Ran Tao, Theo Gevers +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Multimedia · · cited 119× · DOI ↗ · more like this ·

Text in natural images typically adds meaning to an object or scene. In particular, text specifies which business places serve drinks (e.g., cafe, teahouse) or food (e.g., restaurant, pizzeria), and what kind of service is provided (e.g., massage, repair). The mere presence of text, its words, and meaning are closely related to the semantics of the object or scene. This paper exploits textual contents in images for fine-grained business place classification and logo retrieval. There are four main contributions. First, we show that the textual cues extracted by the proposed method are effective for the two tasks. Combining the proposed textual and visual cues outperforms visual only classification and retrieval by a large margin. Second, to extract the textual cues, a generic and fully unsupervised word box proposal method is introduced. The method reaches state-of-the-art word detection…

A Spatially Constrained Generative Model and an EM Algorithm for Image Segmentation

A. Diplaros, Nikos Vlassis, Theo Gevers

Artificial Intelligence · IEEE Transactions on Neural Networks · · cited 115× · DOI ↗ · more like this ·

In this paper, we present a novel spatially constrained generative model and an expectation-maximization (EM) algorithm for model-based image segmentation. The generative model assumes that the unobserved class labels of neighboring pixels in the image are generated by prior distributions with similar parameters, where similarity is defined by entropic quantities relating to the neighboring priors. In order to estimate model parameters from observations, we derive a spatially constrained EM algorithm that iteratively maximizes a lower bound on the data log-likelihood, where the penalty term is data-dependent. Our algorithm is very easy to implement and is similar to the standard EM algorithm for Gaussian mixtures with the main difference that the labels posteriors are "smoothed" over pixels between each E- and M-step by a standard image filter. Experiments on synthetic and real images…

The MediaMill TRECVID 2004 Semantic Video Search Engine

Cees G. M. Snoek, Koen E. A. van de Sande, Ork de Rooij +8 more

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 115× · OpenAlex ↗ · more like this ·

In this paper we describe our TRECVID 2008 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Rather than continuing to increase the number of concept detectors available for retrieval, our TRECVID 2008 experiments focus on increasing the robustness of a small set of detectors. To that end, our concept detection experiments emphasize in particular the role of sampling, the value of color invariant features, the influence of codebook construction, and the effectiveness of kernel-based learning parameters. For retrieval, a robust but limited set of concept detectors necessitates the need to rely on as many auxiliary information channels as possible. Therefore, our automatic search experiments focus on predicting which information channel to trust given a certain topic, leading to a novel framework for…

Like Father, Like Son: Facial Expression Dynamics for Kinship Verification

Hamdi Dibeklioğlu, Albert Ali Salah, Theo Gevers

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 112× · DOI ↗ · more like this ·

Kinship verification from facial appearance is a difficult problem. This paper explores the possibility of employing facial expression dynamics in this problem. By using features that describe facial dynamics and spatio-temporal appearance over smile expressions, we show that it is possible to improve the state of the art in this problem, and verify that it is indeed possible to recognize kinship by resemblance of facial expressions. The proposed method is tested on different kin relationships. On the average, 72.89% verification accuracy is achieved on spontaneous smiles.

Combining Facial Dynamics With Appearance for Age Estimation

Hamdi Dibeklioğlu, Fares Alnajar, Albert Ali Salah +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 110× · DOI ↗ · more like this ·

Estimating the age of a human from the captured images of his/her face is a challenging problem. In general, the existing approaches to this problem use appearance features only. In this paper, we show that in addition to appearance information, facial dynamics can be leveraged in age estimation. We propose a method to extract and use dynamic features for age estimation, using a person's smile. Our approach is tested on a large, gender-balanced database with 400 subjects, with an age range between 8 and 76. In addition, we introduce a new database on posed disgust expressions with 324 subjects in the same age range, and evaluate the reliability of the proposed approach when used with another expression. State-of-the-art appearance-based age estimation methods from the literature are implemented as baseline. We demonstrate that for each of these methods, the addition of the proposed…

Recognition of Genuine Smiles

Hamdi Dibeklioğlu, Albert Ali Salah, Theo Gevers

Computer Vision and Pattern Recognition · IEEE Transactions on Multimedia · · cited 104× · DOI ↗ · more like this ·

Automatic distinction between genuine (spontaneous) and posed expressions is important for visual analysis of social signals. In this paper, we describe an informative set of features for the analysis of face dynamics, and propose a completely automatic system to distinguish between genuine and posed enjoyment smiles. Our system incorporates facial landmarking and tracking, through which features are extracted to describe the dynamics of eyelid, cheek, and lip corner movements. By fusing features over different regions, as well as over different temporal phases of a smile, we obtain a very accurate smile classifier. We systematically investigate age and gender effects, and establish that age-specific classification significantly improves the results, even when the age is automatically estimated. We evaluate our system on the 400-subject UvA-NEMO database we have recently collected, as…

Authentic facial expression analysis

Nicu Sebe, Michael S. Lew, Ira L. Cohen +3 more

Computer Vision and Pattern Recognition · Institutional Research Information System (Università degli Studi di Trento) · · cited 101× · DOI ↗ · more like this ·

It is argued that for the computer to be able to interact with humans, it needs to havve the communication skills o humans. One of these skills is the ability to understand the emotional state of the person. The most expressive way humans display emotions is through facial expressions. In most facial expression systems and databases, the emotion data was collected by asking the subjects to perform a series of facial expressions. However, these directed or deliberate facial action tasks typically differ in appearance and timing from the authentic facial expressions induced through events in the normal environment of the subject. In this paper, we present our effort in creating an authentic facial expression database based on spontaneous emotions derived from the environment. Furthermore, we test and compare a wide range of classifiers from the machine learning literature that can be used…

3D Scene priors for road detection

Jose M. Álvarez, Theo Gevers, Antonio M. López

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 97× · DOI ↗ · more like this ·

Vision-based road detection is important in different areas of computer vision such as autonomous driving, car collision warning and pedestrian crossing detection. However, current vision-based road detection methods are usually based on low-level features and they assume structured roads, road homogeneity, and uniform lighting conditions. Therefore, in this paper, contextual 3D information is used in addition to low-level cues. Low-level photometric invariant cues are derived from the appearance of roads. Contextual cues used include horizon lines, vanishing points, 3D scene layout and 3D road stages. Moreover, temporal road cues are included. All these cues are sensitive to different imaging conditions and hence are considered as weak cues. Therefore, they are combined to improve the overall performance of the algorithm. To this end, the low-level, contextual and temporal cues are…

Learning Content-enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation

Qi Bi, Shaodi You, Theo Gevers

cs.CV · · cited 91× · arXiv ↗ · more like this ·

Domain-generalized urban-scene semantic segmentation (USSS) aims to learn generalized semantic predictions across diverse urban-scene styles. Unlike domain gap challenges, USSS is unique in that the semantic categories are often similar in different urban scenes, while the styles can vary significantly due to changes in urban landscapes, weather conditions, lighting, and other factors. Existing approaches typically rely on convolutional neural networks (CNNs) to learn the content of urban scenes. In this paper, we propose a Content-enhanced Mask TransFormer (CMFormer) for domain-generalized USSS. The main idea is to enhance the focus of the fundamental component, the mask attention mechanism, in Transformer segmentation models on content information. To achieve this, we introduce a novel content-enhanced mask attention mechanism. It learns mask queries from both the image feature and…

A comparison of color features for visual concept classification

Koen E. A. van de Sande, Theo Gevers, Cees G. M. Snoek

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 91× · DOI ↗ · more like this ·

Concept classification is important to access visual information on the level of objects and scene types. So far, intensity-based features have been widely used. To increase discriminative power, color features have been proposed only recently. As many features exist, a structured overview is required of color features in the context of concept classification.

Empowering Visual Categorization With the GPU

Koen E. A. van de Sande, Theo Gevers, Cees G. M. Snoek

Computer Vision and Pattern Recognition · IEEE Transactions on Multimedia · · cited 90× · DOI ↗ · more like this ·

Visual categorization is important to manage large collections of digital images and video, where textual metadata is often incomplete or simply unavailable. The bag-of-words model has become the most powerful method for visual categorization of images and video. Despite its high accuracy, a severe drawback of this model is its high computational cost. As the trend to increase computational power in newer CPU and GPU architectures is to increase their level of parallelism, exploiting this parallelism becomes an important direction to handle the computational cost of the bag-of-words approach. When optimizing a system based on the bag-of-words approach, the goal is to minimize the time it takes to process batches of images. this paper, we analyze the bag-of-words model for visual categorization in terms of computational cost and identify two major bottlenecks: the quantization step and…

Robust photometric invariant features from the color tensor

Joost van de Weijer, Theo Gevers, A.W.M. Smeulders

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 87× · DOI ↗ · more like this ·

Luminance-based features are widely used as low-level input for computer vision applications, even when color data is available. The extension of feature detection to the color domain prevents information loss due to isoluminance and allows us to exploit the photometric information. To fully exploit the extra information in the color data, the vector nature of color data has to be taken into account and a sound framework is needed to combine feature and photometric invariance theory. In this paper, we focus on the structure tensor, or color tensor, which adequately handles the vector nature of color images. Further, we combine the features based on the color tensor with photometric invariant derivatives to arrive at photometric invariant features. We circumvent the drawback of unstable photometric invariants by deriving an uncertainty measure to accompany the photometric invariant…

Towards Personalised Gaming via Facial Expression Recognition

Paris Mavromoustakos Blom, Sander Bakkes, Chek Tien Tan +4 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 80× · DOI ↗ · more like this ·

In this paper we propose an approach for personalising the space in which a game is played (i.e., levels) dependent on classifications of the user's facial expression — to the end of tailoring the affective game experience to the individual user. Our approach is aimed at online game personalisation, i.e., the game experience is personalised during actual play of the game. A key insight of this paper is that game personalisation techniques can leverage novel computer vision-based techniques to unobtrusively infer player experiences automatically based on facial expression analysis. Specifically, to the end of tailoring the affective game experience to the individual user, in this paper we (1) leverage the proven InSight facial expression recognition SDK as a model of the user's affective state InSight, and (2) employ this model for guiding the online game personalisation process. User…

Evaluation of Color STIPs for Human Action Recognition

I. Everts, Jan van Gemert, Theo Gevers

Computer Vision and Pattern Recognition · UvA-DARE (University of Amsterdam) · · cited 71× · DOI ↗ · more like this ·

This paper is concerned with recognizing realistic human actions in videos based on spatio-temporal interest points (STIPs). Existing STIP-based action recognition approaches operate on intensity representations of the image data. Because of this, these approaches are sensitive to disturbing photometric phenomena such as highlights and shadows. Moreover, valuable information is neglected by discarding chromaticity from the photometric representation. These issues are addressed by Color STIPs. Color STIPs are multi-channel reformulations of existing intensity-based STIP detectors and descriptors, for which we consider a number of chromatic representations derived from the opponent color space. This enhanced modeling of appearance improves the quality of subsequent STIP detection and description. Color STIPs are shown to substantially outperform their intensity-based counterparts on the…

Joint Attention by Gaze Interpolation and Saliency

Zeynep Yücel, Albert Ali Salah, Çetin Meriçli +3 more

Human-Computer Interaction · IEEE Transactions on Cybernetics · · cited 71× · DOI ↗ · more like this ·

Joint attention, which is the ability of coordination of a common point of reference with the communicating party, emerges as a key factor in various interaction scenarios. This paper presents an image-based method for establishing joint attention between an experimenter and a robot. The precise analysis of the experimenter's eye region requires stability and high-resolution image acquisition, which is not always available. We investigate regression-based interpolation of the gaze direction from the head pose of the experimenter, which is easier to track. Gaussian process regression and neural networks are contrasted to interpolate the gaze direction. Then, we combine gaze interpolation with image-based saliency to improve the target point estimates and test three different saliency schemes. We demonstrate the proposed method on a human-robot interaction scenario. Cross-subject…

Evaluation of Color Spatio-Temporal Interest Points for Human Action Recognition

I. Everts, Jan van Gemert, Theo Gevers

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · · cited 67× · DOI ↗ · more like this ·

This paper considers the recognition of realistic human actions in videos based on spatio-temporal interest points (STIPs). Existing STIP-based action recognition approaches operate on intensity representations of the image data. Because of this, these approaches are sensitive to disturbing photometric phenomena, such as shadows and highlights. In addition, valuable information is neglected by discarding chromaticity from the photometric representation. These issues are addressed by color STIPs. Color STIPs are multichannel reformulations of STIP detectors and descriptors, for which we consider a number of chromatic and invariant representations derived from the opponent color space. Color STIPs are shown to outperform their intensity-based counterparts on the challenging UCF sports, UCF11 and UCF50 action recognition benchmarks by more than 5% on average, where most of the gain is due…

Saliency of color image derivatives: a comparison between computational models and human perception

Eduard Vazquez, Theo Gevers, Marcel P. Lucassen +2 more

Computer Vision and Pattern Recognition · Journal of the Optical Society of America A · · cited 66× · DOI ↗ · more like this ·

In this paper, computational methods are proposed to compute color edge saliency based on the information content of color edges. The computational methods are evaluated on bottom-up saliency in a psychophysical experiment, and on a more complex task of salient object detection in real-world images. The psychophysical experiment demonstrates the relevance of using information theory as a saliency processing model and that the proposed methods are significantly better in predicting color saliency (with a human-method correspondence up to 74.75% and an observer agreement of 86.8%) than state-of-the-art models. Furthermore, results from salient object detection confirm that an early fusion of color and contrast provide accurate performance to compute visual saliency with a hit rate up to 95.2%.

PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition

Partha Das, Sezer Karaoglu, Theo Gevers

cs.CV · · cited 59× · arXiv ↗ · more like this ·

Intrinsic image decomposition is the process of recovering the image formation components (reflectance and shading) from an image. Previous methods employ either explicit priors to constrain the problem or implicit constraints as formulated by their losses (deep learning). These methods can be negatively influenced by strong illumination conditions causing shading-reflectance leakages. Therefore, in this paper, an end-to-end edge-driven hybrid CNN approach is proposed for intrinsic image decomposition. Edges correspond to illumination invariant gradients. To handle hard negative illumination transitions, a hierarchical approach is taken including global and local refinement layers. We make use of attention layers to further strengthen the learning process. An extensive ablation study and large scale experiments are conducted showing that it is beneficial for edge-driven hybrid IID…