A4-Agent: 一种用于零样本 affordance 推理的代理框架
A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning
AI总结：
A4-Agent 提出一种无需训练的代理框架，通过三个阶段的流水线实现零样本 affordance 推理，优于现有监督方法。
摘要：
基于语言指令识别物体上的交互区域是具身人工智能中的关键问题。现有的端到端模型将高层推理和低层定位整合到单一的流水线中，并依赖于在标注数据集上训练，导致在新物体和未见环境上泛化能力差。在本文中，我们超越了这一范式，提出 A4-Agent，一种无需训练的代理框架，将 affordance 推理分解为三个阶段的流水线。我们的框架在测试时协调专门的基模型：(1) 一个 Dreamer，利用生成模型来可视化交互会如何表现；(2) 一个 Thinker，利用大视觉-语言模型来决定要与哪个物体部分进行交互；(3) 一个 Spotter，协调视觉基模型来精确定位交互区域在哪里。通过利用预训练模型的互补优势，无需任何任务特定的微调，我们的零样本框架在多个基准上显著优于最先进的监督方法，并在现实世界设置中表现出鲁棒的泛化能力。
英文摘要：
Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI. Prevailing end-to-end models couple high-level reasoning and low-level grounding into a single monolithic pipeline and rely on training over annotated datasets, which leads to poor generalization on novel objects and unseen environments. In this paper, we move beyond this paradigm by proposing A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline. Our framework coordinates specialized foundation models at test time: (1) a $\textbf{Dreamer}$ that employs generative models to visualize $\textit{how}$ an interaction would look; (2) a $\textbf{Thinker}$ that utilizes large vision-language models to decide $\textit{what}$ object part to interact with; and (3) a $\textbf{Spotter}$ that orchestrates vision foundation models to precisely locate $\textit{where}$ the interaction area is. By leveraging the complementary strengths of pre-trained models without any task-specific fine-tuning, our zero-shot framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks and demonstrates robust generalization to real-world settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
