用于弱监督可供性定位的选择性对比学习
Selective Contrastive Learning for Weakly Supervised Affordance Grounding
AI总结：
针对弱监督可供性定位中模型易聚焦无关模式的问题，提出选择性原型与像素对比目标，在部位和物体层面自适应学习可供性线索，有效将激活转向有意义区域。
摘要：
促进实体与物体的交互需要准确识别可供特定动作执行的部分。弱监督可供性定位（WSAG）旨在模仿人类从第三人称演示中学习的能力，人类能直觉地掌握功能部位而无需像素级标注。为实现这一目标，通常通过在不同视角图像间共享分类器，并结合引入部位发现过程的蒸馏策略来学习定位。然而，由于可供性相关部位并不总是易于区分，模型主要依赖分类，往往聚焦于与可供性无关的常见类别特定模式。为解决该局限，我们引入选择性原型与像素对比目标，根据可用信息的粒度，在部位和物体层面自适应地学习可供性相关线索，从而超越了孤立的部位级学习。最初，我们利用CLIP在自我中心（物体聚焦）和外部中心（第三人称示例）图像中寻找动作关联物体。随后，通过交叉比对互补视图发现的物体，我们在每个视角中挖掘精确的部位级可供性线索。通过持续学习区分可供性相关区域与可供性无关的背景上下文，我们的方法有效地将激活从无关区域转向有意义的可供性线索。实验结果证明了我们方法的有效性。代码可在github.com/hynnsk/SelectiveCL获取。
英文摘要：
Facilitating an entity's interaction with objects requires accurately identifying parts that afford specific actions. Weakly supervised affordance grounding (WSAG) seeks to imitate human learning from third-person demonstrations, where humans intuitively grasp functional parts without needing pixel-level annotations. To achieve this, grounding is typically learned using a shared classifier across images from different perspectives, along with distillation strategies incorporating part discovery process. However, since affordance-relevant parts are not always easily distinguishable, models primarily rely on classification, often focusing on common class-specific patterns that are unrelated to affordance. To address this limitation, we move beyond isolated part-level learning by introducing selective prototypical and pixel contrastive objectives that adaptively learn affordance-relevant cues at both the part and object levels, depending on the granularity of the available information. Initially, we find the action-associated objects in both egocentric (object-focused) and exocentric (third-person example) images by leveraging CLIP. Then, by cross-referencing the discovered objects of complementary views, we excavate the precise part-level affordance clues in each perspective. By consistently learning to distinguish affordance-relevant regions from affordance-irrelevant background context, our approach effectively shifts activation from irrelevant areas toward meaningful affordance cues. Experimental results demonstrate the effectiveness of our method. Codes are available at github.com/hynnsk/SelectiveCL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
