arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

David Varas

15 papers · 2009–2022

2009: 12010: 02011: 02012: 12013: 02014: 22015: 12016: 02017: 12018: 22019: 22020: 22021: 22022: 1
2009peak 2 in 20142022

Frequent co-authors

Papers

showing 1–15 of 15 RSS

Region-Based Particle Filter for Video Object Segmentation

David Varas, Ferran Marqués

Computer Vision and Pattern Recognition · UPCommons institutional repository (Universitat Politècnica de Catalunya) · · cited 33× · DOI ↗ · more like this ·

We present a video object segmentation approach that extends the particle filter to a region-based image representation. Image partition is considered part of the particle filter measurement, which enriches the available information and leads to a re-formulation of the particle filter. The prediction step uses a co-clustering between the previous image object partition and a partition of the current one, which allows us to tackle the evolution of non-rigid structures. Particles are defined as unions of regions in the current image partition and their propagation is computed through a single co-clustering. The proposed technique is assessed on the SegTrack dataset, leading to satisfactory perceptual results and obtaining very competitive pixel error rates compared with the state-of-the-art methods.

Multiresolution hierarchy co-clustering for semantic segmentation in sequences with small variations

David Varas, Mónica Alfaro, Ferran Marques

cs.CV · · cited 19× · arXiv ↗ · more like this ·

This paper presents a co-clustering technique that, given a collection of images and their hierarchies, clusters nodes from these hierarchies to obtain a coherent multiresolution representation of the image collection. We formalize the co-clustering as a Quadratic Semi-Assignment Problem and solve it with a linear programming relaxation approach that makes effective use of information from hierarchies. Initially, we address the problem of generating an optimal, coherent partition per image and, afterwards, we extend this method to a multiresolution framework. Finally, we particularize this framework to an iterative multiresolution video segmentation algorithm in sequences with small variations. We evaluate the algorithm on the Video Occlusion/Object Boundary Detection Dataset, showing that it produces state-of-the-art results in these scenarios.

ViTS: Video Tagging System from Massive Web Multimedia Collections

Delia Fernandez, David Varas, Joan Espadaler +7 more

Computer Vision and Pattern Recognition · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · · cited 17× · DOI ↗ · more like this ·

The popularization of multimedia content on the Web has arised the need to automatically understand, index and retrieve it. In this paper we present ViTS, an automatic Video Tagging System which learns from videos, their web context and comments shared on social networks. ViTS analyses massive multimedia collections by Internet crawling, and maintains a knowledge base that updates in real time with no need of human supervision. As a result, each video is indexed with a rich set of labels and linked with other related contents. ViTS is an industrial product under exploitation with a vocabulary of over 2.5M concepts, capable of indexing more than 150k videos per month. We compare the quality and completeness of our tags with respect to the ones in the YouTube-8M dataset, and we show how ViTS enhances the semantic annotation of the videos with a larger number of labels (10.04 tags/video)…

Multi-Modal Pyramid Feature Combination for Human Action Recognition

Carlos Roig, Manuel Sarmiento, David Varas +3 more

Computer Vision and Pattern Recognition · · cited 4× · DOI ↗ · more like this ·

Accurate human action recognition remains a challenging task in the field of computer vision. While many approaches focus on narrow image features, this work proposes a novel multi-modal method that combines task specific features (action recognition, scene understanding, object detection and acoustic event detection) for human action recognition. This work encompasses two contributions: 1) The introduction of a feature fusion block that uses a gating mechanism to perform attention over features from other domains and 2) A pyramidal feature combination approach that hierarchically combines pairs of features from different tasks using the previous fusion block. The richer features generated by the pyramid are used for human action recognition. This approach is validated using a subset of the Moments In Time dataset, resulting in an accuracy of 35.43%.

A region-based particle filter for generic object tracking and segmentation

David Varas, Ferran Marqués

Computer Vision and Pattern Recognition · · cited 4× · DOI ↗ · more like this ·

In this work we present a region-based particle filter for generic object tracking and segmentation. The representation of the object in terms of regions homogeneous in color allows the proposed algorithm to robustly track the object and accurately segment its shape along the sequence. Moreover, this segmentation provides a mechanism to update the target model and allows the tracker to deal with color and shape variations of the object. The performance of the algorithm has been tested using the LabelMe Video public database. The experiments show satisfactory results in both tracking and segmentation of the object without an important increase of the computational time due to an efficient computation of the image partition.

Smooth Proxy-Anchor Loss for Noisy Metric Learning

Carlos Roig, David Varas, Issey Masuda +2 more

cs.CV · · cited 3× · arXiv ↗ · more like this ·

Many industrial applications use Metric Learning as a way to circumvent scalability issues when designing systems with a high number of classes. Because of this, this field of research is attracting a lot of interest from the academic and non-academic communities. Such industrial applications require large-scale datasets, which are usually generated with web data and, as a result, often contain a high number of noisy labels. While Metric Learning systems are sensitive to noisy labels, this is usually not tackled in the literature, that relies on manually annotated datasets. In this work, we propose a Metric Learning method that is able to overcome the presence of noisy labels using our novel Smooth Proxy-Anchor Loss. We also present an architecture that uses the aforementioned loss with a two-phase learning procedure. First, we train a confidence module that computes sample class…

Linking media: adopting semantic technologies for multimodal media connection

Delia Cabrera DeBuc, Elisenda Bou‐Balust, Xavier Giró-i-Nieto +8 more

Artificial Intelligence · UPCommons institutional repository (Universitat Politècnica de Catalunya) · · cited 3× · OpenAlex ↗ · more like this ·

Today's media and news organizations are constantly generating large amounts of multimedia content, majorly delivered online. As the online media market grows, the management and delivery of contents is becoming a challenge. Computational approaches can help to overcome this challenge by governing different applications such as content creation, production, search, and its promotion and distribution to different audiences. In this abstract we present a success story of the adoption of semantic technologies on the aforementioned applications, which are built on top of a semantic tagging framework, based on a Knowledge Graph (KG). The presented pipeline combines multimodal inputs into a contextual entity linking module, which indexes documents and links them to trends and stories developing on the news. We will describe how documents are linked and provided to media producers through…

3D shape reconstruction from a humanoid generated video sequence

Pedro A. Martinez, David Varas, Mario Castelán +3 more

Computer Vision and Pattern Recognition · · cited 3× · DOI ↗ · more like this ·

This paper presents a strategy for estimating the geometry of an interest object from a monocular video sequence acquired by a walking humanoid robot. The problem is solved using a space carving algorithm, which relies on both the accurate extraction of the occluding boundaries of the object as well as the precise estimation of the camera pose for each video frame. For data acquisition, a monocular visual-based control has been developed that drives the trajectory of the robot around an object placed on a small table. Due to the stepping of the humanoid, the recorded sequence is contaminated with artefacts that affect the correct extraction of contours along the video frames. To overcome this issue, a method that assigns a fitness score for each frame is proposed, delivering a subset of camera poses and video frames that produce consistent 3D shape estimations of the objects used for…

Generalized Local Attention Pooling for Deep Metric Learning

Carlos Roig, David Varas, Issey Masuda +2 more

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

Deep metric learning has been key to recent advances in face verification and image retrieval amongst others. These systems consist on a feature extraction block (extracts feature maps from images) followed by a spatial dimensionality reduction block (generates compact image representations from the feature maps) and an embedding generation module (projects the image representation to the embedding space). While research on deep metric learning has focused on improving the losses for the embedding generation module, the dimensionality reduction block has been overlooked. In this work, we propose a novel method to generate compact image representations which uses local spatial information through an attention mechanism, named Generalized Local Attention Pooling (GLAP). This method, instead of being placed at the end layer of the backbone, is connected at an intermediate level, resulting…

Unsupervised Multi-label Dataset Generation from Web Data

Carlos Roig, David Varas, Issey Masuda +2 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

This paper presents a system towards the generation of multi-label datasets from web data in an unsupervised manner. To achieve this objective, this work comprises two main contributions, namely: a) the generation of a low-noise unsupervised single-label dataset from web-data, and b) the augmentation of labels in such dataset (from single label to multi label). The generation of a single-label dataset uses an unsupervised noise reduction phase (clustering and selection of clusters using anchors) obtaining a 85% of correctly labeled images. An unsupervised label augmentation process is then performed to assign new labels to the images in the dataset using the class activation maps and the uncertainty associated with each class. This process is applied to the dataset generated in this paper and a public dataset (Places365) achieving a 9.5% and 27% of extra labels in each dataset…

Face tracking using a region-based mean-shift algorithm with adaptive object and background models

Verónica Vilaplana, David Varas

Computer Vision and Pattern Recognition · UPCommons institutional repository (Universitat Politècnica de Catalunya) · · cited 2× · DOI ↗ · more like this ·

This paper proposes a technique for face tracking based on the mean shift algorithm and the segmentation of the images into regions homogeneous in color. Object and background are explicitly modeled and updated through the tracking process. Color and shape information are used to define with precision the face contours, providing a mechanism to adapt the tracker to variations in object scale and to illumination and background changes.

Multi-Scale Transformer-Based Feature Combination for Image Retrieval

Carlos Roig Mari, David Varas González, Elisenda Bou‐Balust

Computer Vision and Pattern Recognition · 2022 IEEE International Conference on Image Processing (ICIP) · · cited 1× · DOI ↗ · more like this ·

Image retrieval consists in the selection of a set of images from a database, based on their visual similarity to a given query image. To compute this visual similarity, a global or multiple local features are required per image.In this work, we present a system that uses multiple high-level semantic feature maps from a single image for global feature generation. Our method uses feature maps extracted from a backbone architecture at different spatial resolutions, enhancing the semantic information relevant at each scale using a transformer-based approach. Then, these feature maps are combined using a self-attention mechanism generating a global feature.The main contribution of this work is a novel global feature generation method, which outperforms current state-of-the-art techniques that use either global-only or a combination of global and local features. This result is assessed using…

Spatio-Temporal Context for Action Detection

Manuel Sarmiento Calderó, David Varas, Elisenda Bou-Balust

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Research in action detection has grown in the recentyears, as it plays a key role in video understanding. Modelling the interactions (either spatial or temporal) between actors and their context has proven to be essential for this task. While recent works use spatial features with aggregated temporal information, this work proposes to use non-aggregated temporal information. This is done by adding an attention based method that leverages spatio-temporal interactions between elements in the scene along the clip.The main contribution of this work is the introduction of two cross attention blocks to effectively model the spatial relations and capture short range temporal interactions.Experiments on the AVA dataset show the advantages of the proposed approach that models spatio-temporal relations between relevant elements in the scene, outperforming other methods that model actor…

3D hierarchical optimization for Multi-view depth map coding

Marc Maceira, David Varas, Josep-Ramon Morros +2 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Depth data has a widespread use since the popularity of high-resolution 3D sensors. In multi-view sequences, depth information is used to supplement the color data of each view. This article proposes a joint encoding of multiple depth maps with a unique representation. Color and depth images of each view are segmented independently and combined in an optimal Rate-Distortion fashion. The resulting partitions are projected to a reference view where a coherent hierarchy for the multiple views is built. A Rate-Distortionoptimization is applied to obtain the final segmentation choosing nodes of the hierarchy. The consistent segmentation is used to robustly encode depth maps of multiple views obtaining competitive results with HEVC coding standards. Available at: http://link.springer.com/article/10.1007/s11042-017-5409-z

What Is Going on in the World? A Display Platform for Media Understanding

Delia Fernandez, Joan Espadaler, David Varas +8 more

Computer Vision and Pattern Recognition · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · · cited 0× · DOI ↗ · more like this ·

News broadcasters and on-line publishers daily generate a large amount of articles and videos describing events currently happening in the world. In this work, we present a system that automatically indexes videos from a library and links them to stories developing in the news. The user interface displays in an intuitive manner the links between videos and stories and allows navigation through related content by using associated tags. This interface is a powerful industrial tool for publishers to index, retrieve and visualize their video content. It helps them identify which topics require more attention or retrieve related content that has already been published about the stories.