arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Vision and Pattern Recognition

To Appear In Improving Video Activity Recognition using Object Recognition and Text Mining

Tanvi S. Motwani, Raymond J. Mooney

See on the map

Abstract

Abstract. Recognizing activities in real-world videos is a challenging AI problem. We present a novel combination of standard activity classification, object recognition, and text mining to learn effective activity recognizers without ever explicitly labeling training videos. We cluster verbs used to describe videos to automatically discover classes of activities and produce a labeled training set. This labeled data is then used to train an activity classifier based on spatiotemporal features. Next, text mining is employed to learn the correlations between these verbs and related objects. This knowledge is then used together with the outputs of an off-the-shelf object recognizer and the trained activity classifier to produce an improved activity recognizer. Experiments on a corpus of YouTube videos demonstrate the effectiveness of the overall approach. with natural-language descriptions [3] verify that our approach improves the accuracy of a standard activity recognizer for real-world videos. Figure 1 shows sample frames from a couple of videos with their linguistic descriptions. The remainder of the paper is organized as follows. Section 2 discusses related work, Section 3 describes our new system, Section 4 experimentally evaluates it on real-world videos, Section 5 discusses future work and Section 6 presents our conclusions. 1

More like this