INTRA：交互关系感知的弱监督可供性定位
INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding
AI总结：
本文提出 INTRA，仅用外中心图像通过对比学习和视觉-语言模型嵌入实现文本条件化弱监督可供性定位，在多个数据集上优于已有方法并具备良好泛化能力。
摘要：
Affordance（可供性）表示物体固有的潜在交互方式。对可供性的感知能够使智能体在新环境中高效导航并进行交互。弱监督可供性定位无需昂贵的像素级标注，而是借助外中心图像来教授智能体可供性概念。尽管近期弱监督可供性定位的进展取得了可观成果，但仍存在挑战，包括需要成对的外中心与自我中心图像数据集，以及为单个物体定位多种可供性的复杂性。为解决这些问题，我们提出了交互关系感知的弱监督可供性定位方法 INTRA。与以往方法不同，INTRA 将该问题重构为表征学习，仅利用外中心图像通过对比学习识别交互的独特特征，从而消除了对成对数据集的需求。此外，我们利用视觉-语言模型嵌入，灵活地基于任意文本执行可供性定位，设计了文本条件化的可供性图生成，以在对比学习中反映交互关系，并通过文本同义词增强提升鲁棒性。我们的方法在 AGD20K、IIT-AFF、CAD 和 UMD 等多种数据集上优于以往方法。此外，实验结果表明，该方法对合成图像/插图具有显著的领域可扩展性，并能够对新颖交互和物体执行可供性定位。
英文摘要：
Affordance denotes the potential interactions inherent in objects. The perception of affordance can enable intelligent agents to navigate and interact with new environments efficiently. Weakly supervised affordance grounding teaches agents the concept of affordance without costly pixel-level annotations, but with exocentric images. Although recent advances in weakly supervised affordance grounding yielded promising results, there remain challenges including the requirement for paired exocentric and egocentric image dataset, and the complexity in grounding diverse affordances for a single object. To address them, we propose INTeraction Relationship-aware weakly supervised Affordance grounding (INTRA). Unlike prior arts, INTRA recasts this problem as representation learning to identify unique features of interactions through contrastive learning with exocentric images only, eliminating the need for paired datasets. Moreover, we leverage vision-language model embeddings for performing affordance grounding flexibly with any text, designing text-conditioned affordance map generation to reflect interaction relationship for contrastive learning and enhancing robustness with our text synonym augmentation. Our method outperformed prior arts on diverse datasets such as AGD20K, IIT-AFF, CAD and UMD. Additionally, experimental results demonstrate that our method has remarkable domain scalability for synthesized images / illustrations and is capable of performing affordance grounding for novel interactions and objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
