arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.08453cs.CV
收藏

基于姿态感知的多层次特征网络的人机交互检测

Pose-aware Multi-level Feature Network for Human Object Interaction Detection

ShanghaiTech University(上海科技大学)

Bo Wan, Desen Zhou, Yongfei Liu, Rongjie Li, Xuming He

摘要:

推理人与物体的交互关系是以人为中心的场景理解的核心问题,由于人与物体之间的空间结构变化大、关系实例多、关系类别之间存在细微的视觉差异,检测人与物体之间的交互关系对视觉系统提出了独特的挑战。为了解决这些挑战,我们提出了一个多层次的关系检测策略,利用人体姿势线索捕捉全球空间配置的关系,并作为一个注意力机制,动态放大到相关区域的人体部位水平。具体来说,我们开发了一个多分支深度网络,在三个语义层次上学习姿势增强关系表示,结合交互上下文,对象特征和详细的语义部分线索。因此,我们的方法是能够产生强大的预测细粒度的人类对象的相互作用与可解释的输出。在公共基准上进行的广泛的实验评估表明,我们的模型在相当大的范围内优于先前的方法,证明了它在处理复杂场景方面的有效性。

英文摘要:

Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories. To address those challenges, we propose a multi-level relation detection strategy that utilizes human pose cues to capture global spatial configurations of relations and as an attention mechanism to dynamically zoom into relevant regions at human part level. Specifically, we develop a multi-branch deep network to learn a pose-augmented relation representation at three semantic levels, incorporating interaction context, object features and detailed semantic part cues. As a result, our approach is capable of generating robust predictions on fine-grained human object interactions with interpretable outputs. Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑