arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Vision and Pattern Recognition

Spatio-Temporal Context of Mid-level Features for Activity Recognition

Fei Yuan, Gui-Song Xia, Hichem Sahbi, V. Prinet

See on the map

Abstract

Local spatio-temporal features have been shown to be ecient and robust to represent simple actions. However, for complicated human activities with long-range motion or multiple interactive body parts and persons, the limitation of low-level features blows up because of their local properties and the lack of context. This paper addresses the problem by suggesting a framework for both computing mid-level features and taking into account the context information between them. First, we propose to represent human activities by a set of mid-level components, called activity components, which have consistent structure and motion in spatial and temporal domain respectively. The activity components are extracted hierarchically from videos, i.e., extracting key-points, grouping key-points to trajectories and nally clustering trajectories to components. Second, to further exploit the interdependencies of the activity components, we introduce a spatio-temporal context kernel (STCK), which not only takes into account the local properties of features but also considers their spatial and temporal context information. The experimental results on two challenging datasets of activities show that the proposed activity components outperform local spatio-temporal features and the STCK improves a lot the performance of activity recognition by considering the context between activity components.

More like this