arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.08967cs.CV
收藏

基于交互身体部位注意的人机交互识别框架

Human Interaction Recognition Framework based on Interacting Body Part Attention

Kyungpook National University(庆北国立大学);Korea University(高丽大学)

Dong-Gyu Lee, Seong-Whan Lee

摘要:

视频中的人类活动识别已经得到了广泛的研究,最近通过深度学习方法取得了重大进展;然而,它仍然是一项具有挑战性的任务。在本文中,我们提出了一个新的框架,同时考虑隐式和显式表示的人类互动的融合信息的局部图像的互动积极发生,原始运动与个人主体的身体部位的姿势,并共同出现的整体外观变化。人与人之间的互动会发生变化,这取决于每个人的身体部位如何与另一个人互动。所提出的方法捕捉使用相互作用的身体部位的注意力不同的相互作用之间的细微差别。与其他对象交互的语义重要的身体部位在特征表示期间被赋予更多权重。将交互的基于身体部位注意力的个体表示和全身外观变化的同现描述符的组合特征馈送到长短期记忆中,以在单个框架中对随时间的时间动态进行建模。我们使用四个广泛使用的公共数据集验证所提出的方法的有效性,优于竞争国家的最先进的方法。

英文摘要:

Human activity recognition in videos has been widely studied and has recently gained significant advances with deep learning approaches; however, it remains a challenging task. In this paper, we propose a novel framework that simultaneously considers both implicit and explicit representations of human interactions by fusing information of local image where the interaction actively occurred, primitive motion with the posture of individual subject's body parts, and the co-occurrence of overall appearance change. Human interactions change, depending on how the body parts of each human interact with the other. The proposed method captures the subtle difference between different interactions using interacting body part attention. Semantically important body parts that interact with other objects are given more weight during feature representation. The combined feature of interacting body part attention-based individual representation and the co-occurrence descriptor of the full-body appearance change is fed into long short-term memory to model the temporal dynamics over time in a single framework. We validate the effectiveness of the proposed method using four widely used public datasets by outperforming the competing state-of-the-art method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑