探索并弥合几何-交互线索以实现视觉基础模型中的可及性推理
Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models
AI总结：
本文探讨了视觉基础模型中几何感知与交互感知对可及性推理的互补作用，通过融合DINO的几何原型与Flux的交互地图，实现了与弱监督方法相当的可及性估计。
摘要：
什么是视觉系统真正理解可及性意味着什么？我们主张这种理解取决于两种互补能力：几何感知，它识别使交互可能的物体结构部分，以及交互感知，它建模代理动作如何与这些部分互动。为了验证这一假设，我们系统地测试了视觉基础模型（VFMs）。我们发现，像DINO这样的模型本质上编码了部分级几何结构，而生成模型如Flux包含丰富的、动词条件的空问注意力图，这些图充当隐含的交互先验。关键的是，我们证明这两种维度不仅仅是相关的，而是可及性可组合的要素。通过在无训练和零样本的方式下融合DINO的几何原型与Flux的交互地图，我们实现了与弱监督方法相当的可及性估计。最终融合实验确认了几何感知和交互感知是视觉基础模型中可及性理解的基本构建块，提供了感知如何引导行动的机制性解释。
英文摘要：
What does it mean for a visual system to truly understand affordance? We argue that this understanding hinges on two complementary capacities: geometric perception, which identifies the structural parts of objects that enable interaction, and interaction perception, which models how an agent's actions engage with those parts. To test this hypothesis, we conduct a systematic probing of Visual Foundation Models (VFMs). We find that models like DINO inherently encode part-level geometric structures, while generative models like Flux contain rich, verb-conditioned spatial attention maps that serve as implicit interaction priors. Crucially, we demonstrate that these two dimensions are not merely correlated but are composable elements of affordance. By simply fusing DINO's geometric prototypes with Flux's interaction maps in a training-free and zero-shot manner, we achieve affordance estimation competitive with weakly-supervised methods. This final fusion experiment confirms that geometric and interaction perception are the fundamental building blocks of affordance understanding in VFMs, providing a mechanistic account of how perception grounds action.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
