arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.04743cs.CV
收藏

基于双注意网络的人-物交互推理

Reasoning About Human-Object Interactions Through Dual Attention Networks

University of California, Berkeley(加州大学伯克利分校);MIT-IBM Watson AI Lab, IBM Research(麻省理工学院-IBM沃森人工智能实验室,IBM研究院);Massachusetts Institute of Technology(麻省理工学院);The Chinese University of Hong Kong(香港中文大学)

Tete Xiao, Quanfu Fan, Dan Gutfreund, Mathew Monfort, Aude Oliva, Bolei Zhou

摘要:

对象是我们作用的实体,其中对象的功能取决于我们如何与它交互。在这项工作中,我们提出了一个双注意力网络模型的原因,人与对象的交互。双注意框架分别对物体和动作的重要特征进行加权。因此,对对象和动作的识别是相互受益的。该模型在人-物交互数据集Something-Something上显示出有竞争力的分类性能。此外,它可以执行弱时空定位和启示分割,尽管只使用视频级标签进行训练。该模型不仅可以发现动作何时发生以及哪个对象正在被操作,还可以识别对象的哪个部分正在被交互。项目页面:\url{https://dual-attention-network.github.io/}。

英文摘要:

Objects are entities we act upon, where the functionality of an object is determined by how we interact with it. In this work we propose a Dual Attention Network model which reasons about human-object interactions. The dual-attentional framework weights the important features for objects and actions respectively. As a result, the recognition of objects and actions mutually benefit each other. The proposed model shows competitive classification performance on the human-object interaction dataset Something-Something. Besides, it can perform weak spatiotemporal localization and affordance segmentation, despite being trained only with video-level labels. The model not only finds when an action is happening and which object is being manipulated, but also identifies which part of the object is being interacted with. Project page: \url{https://dual-attention-network.github.io/}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑