通过记忆实现 grounding：跨场景和场景内的记忆用于3D功能 affordances
Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances
AI总结：
本文提出AFFORDMEM框架，通过跨场景和场景内的记忆实现3D功能 affordances的grounding，无需微调和标注，提升AP50性能。
摘要：
功能性 affordance grounding需要超越物体识别：智能体必须定位支持交互的具体区域，如拉把手或按按钮。这在无训练视觉语言管道中具有挑战性，因为可操作区域通常小、视觉模糊且在多个同类别实例中重复出现。我们提出AFFORDMEM框架，通过两个层次的记忆来ground 3D功能 affordances。第一是跨场景affordance记忆：智能体维护一个包含RGB图像的类别级记忆库，其中affordance区域以叠加形式呈现，并在查询时回忆最信息性的示例，引导冻结的VLM朝向文本提示常忽略的小可操作子区域。第二是场景内空间记忆：当智能体处理场景时，它将候选实例及其3D空间关系组织成结构化的场景图，使语言模型能够解决远距离或当前未观察到的候选物的引用，如“从顶部第二个把手”。AFFORDMEM无需模型微调和目标场景标注，使用从源场景构建的可重用记忆库。在SceneFun3D上，我们的方法在Split 0和Split 1上分别比之前的无训练状态的最优方法提高了3.23和3.7。消融研究支持互补的收益：跨场景affordance记忆提高细粒度定位，而场景内空间记忆在空间上合格的查询中提供更大的收益。项目主页可在项目页面上获得。
英文摘要：
Functional affordance grounding requires more than recognizing an object: an agent must localize the specific region that supports an interaction, such as the handle to pull or the button to press. This is difficult for training-free vision-language pipelines because actionable regions are often small, visually ambiguous, and repeated across multiple same-category instances in a scene. We propose AFFORDMEM, a framework that grounds 3D functional affordances by remembering geometry at two levels. The first is cross-scene affordance memory: the agent maintains a category-level memory bank of RGB images with affordance regions rendered as overlays, and recalls the most informative examples at query time to guide a frozen VLM toward small operable subregions that text-only prompting consistently misses. The second is in-scene spatial memory: as the agent processes the scene, it organizes candidate instances and their 3D spatial relations into a structured scene graph, enabling the language model to resolve references over distant or currently unobserved candidates such as "the second handle from the top." AFFORDMEM requires no model fine-tuning and no target-scene annotation, using a reusable memory bank built from source scenes. On SceneFun3D, our method improves AP50 over the prior training-free state of the art by 3.23 on Split 0 and 3.7 on Split 1. Ablation studies support complementary benefits: cross-scene affordance memory improves fine-grained localization, while in-scene spatial memory provides the larger gain on spatially qualified queries. The project homepage is available at the project page.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
