Computer Science · Computer Vision and Pattern Recognition
Spatio-Temporal Context of Mid-level Features for Activity Recognition
Fei Yuan, Gui-Song Xia, Hichem Sahbi, V. Prinet
See on the mapAbstract
Local spatio-temporal features have been shown to be ecient and robust to represent simple actions. However, for complicated human activities with long-range motion or multiple interactive body parts and persons, the limitation of low-level features blows up because of their local properties and the lack of context. This paper addresses the problem by suggesting a framework for both computing mid-level features and taking into account the context information between them. First, we propose to represent human activities by a set of mid-level components, called activity components, which have consistent structure and motion in spatial and temporal domain respectively. The activity components are extracted hierarchically from videos, i.e., extracting key-points, grouping key-points to trajectories and nally clustering trajectories to components. Second, to further exploit the interdependencies of the activity components, we introduce a spatio-temporal context kernel (STCK), which not only takes into account the local properties of features but also considers their spatial and temporal context information. The experimental results on two challenging datasets of activities show that the proposed activity components outperform local spatio-temporal features and the STCK improves a lot the performance of activity recognition by considering the context between activity components.