从第三人称视角进行可操作性定位
Grounded Affordance from Exocentric View
AI总结：
本文提出从第三人称视角进行可操作性定位的任务，设计跨视角知识迁移框架，并构建AGD20K数据集，在客观指标和视觉质量上优于代表性模型。
摘要：
可操作性定位旨在定位物体的“动作可能性”区域，这是实现具身智能的关键一步。由于交互可操作性的多样性，不同个体的独特性导致了多样化的交互，这使得在物体部件与可操作性标签之间建立显式联系变得困难。人类具备将各种第三人称交互转化为不变的第一人称可操作性以应对交互多样性影响的能力。为了赋予智能体这种能力，本文提出了一个从第三人称视角进行可操作性定位的任务，即给定第三人称人-物交互图像和第一人称物体图像，仅使用可操作性标签作为监督，学习物体的可操作性知识并将其迁移到第一人称图像中。然而，不同角色之间存在一些“交互偏差”，主要体现在不同区域和不同视角上。为此，我们设计了一个跨视角可操作性知识迁移框架，从第三人称交互中提取可操作性特定特征，并将其迁移到第一人称视角。具体而言，通过保留可操作性共现关系来增强对可操作性区域的感知。此外，我们构建了一个名为AGD20K的可操作性定位数据集，收集并标注了来自36个可操作性类别的超过20K张图像。实验结果表明，我们的方法在客观指标和视觉质量方面均优于代表性模型。代码已发布于https://github.com/lhc1224/Cross-view-affordance-grounding。
英文摘要：
Affordance grounding aims to locate objects' "action possibilities" regions, which is an essential step toward embodied intelligence. Due to the diversity of interactive affordance, the uniqueness of different individuals leads to diverse interactions, which makes it difficult to establish an explicit link between object parts and affordance labels. Human has the ability that transforms the various exocentric interactions into invariant egocentric affordance to counter the impact of interactive diversity. To empower an agent with such ability, this paper proposes a task of affordance grounding from exocentric view, i.e., given exocentric human-object interaction and egocentric object images, learning the affordance knowledge of the object and transferring it to the egocentric image using only the affordance label as supervision. However, there is some "interaction bias" between personas, mainly regarding different regions and different views. To this end, we devise a cross-view affordance knowledge transfer framework that extracts affordance-specific features from exocentric interactions and transfers them to the egocentric view. Specifically, the perception of affordance regions is enhanced by preserving affordance co-relations. In addition, an affordance grounding dataset named AGD20K is constructed by collecting and labeling over 20K images from $36$ affordance categories. Experimental results demonstrate that our method outperforms the representative models regarding objective metrics and visual quality. Code is released at https://github.com/lhc1224/Cross-view-affordance-grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
