arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1912.07249cs.CV
收藏

模仿:脱离语境理解人类行为

Mimetics: Towards Understanding Human Actions Out of Context

NAVER Labs Europe(NAVER欧洲实验室)

Philippe Weinzaepfel, Grégory Rogez

摘要:

最近的视频动作识别方法在现有的基准上已经达到了出色的性能。然而,他们倾向于利用场景或物体等背景,而不是专注于理解人类行为本身。例如,网球场导致预测打网球,而不管在视频中执行的动作。相比之下,人类对动作有更完整的理解,可以在没有上下文的情况下识别它们。最好的例子是哑剧,人们通常可以识别,尽管缺少相关的对象和场景。在本文中,我们建议在缺乏上下文的情况下对动作识别方法进行基准测试,并引入一个新的数据集Mimetics,由来自Kinetics基准的50个类的子集的模仿动作组成。我们的实验表明,(a)最先进的3D卷积神经网络在此类视频上获得了令人失望的结果,突出了对人类行为缺乏真正的理解,以及(b)通过人体姿势利用肢体语言的模型不太容易出现上下文偏差。特别是,我们表明,应用一个浅神经网络与一个单一的时间卷积的身体姿势特征转移到动作识别问题相比,3D动作识别方法执行令人惊讶的好。

英文摘要:

Recent methods for video action recognition have reached outstanding performances on existing benchmarks. However, they tend to leverage context such as scenes or objects instead of focusing on understanding the human action itself. For instance, a tennis field leads to the prediction playing tennis irrespectively of the actions performed in the video. In contrast, humans have a more complete understanding of actions and can recognize them without context. The best example of out-of-context actions are mimes, that people can typically recognize despite missing relevant objects and scenes. In this paper, we propose to benchmark action recognition methods in such absence of context and introduce a novel dataset, Mimetics, consisting of mimed actions for a subset of 50 classes from the Kinetics benchmark. Our experiments show that (a) state-of-the-art 3D convolutional neural networks obtain disappointing results on such videos, highlighting the lack of true understanding of the human actions and (b) models leveraging body language via human pose are less prone to context biases. In particular, we show that applying a shallow neural network with a single temporal convolution over body pose features transferred to the action recognition problem performs surprisingly well compared to 3D action recognition methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑