SAGA：通过结构化可及性 grounding 实现开放世界移动操作
SAGA: Open-World Mobile Manipulation via Structured Affordance Grounding
AI总结：
SAGA通过结构化可及性接地实现开放世界移动操作，能有效处理多种任务形式并实现零样本执行。
摘要：
我们提出了SAGA，一种多功能且适应性强的视觉-运动控制框架，能够在各种环境、任务目标和用户规范之间进行泛化。为了高效学习这种能力，我们的关键思想是通过显式地将任务目标接地在观察到的环境中，将高层语义意图与低层视觉-运动控制解耦。使用基于可及性的任务表示，我们以统一的结构形式表达多样且复杂的动作。通过利用多模态基础模型，SAGA将所提出的任务表示接地到机器人的视觉观察作为3D可及性热图，突出任务相关实体，同时抽象掉会阻碍泛化的虚假外观变化。这些接地的可及性使我们能够有效地在多任务演示数据上训练一个条件策略，用于全身控制。在一个统一的框架中，SAGA可以解决以不同形式指定的任务，包括语言指令、选定点和示例演示，实现零样本执行和少样本适应。我们将在四足操作器上实例化SAGA，并在十一项现实任务上进行广泛实验。SAGA在端到端和模块化基线方面表现显著优越。这些结果共同表明，结构化可及性接地为通用移动操作提供了一条可扩展且有效的路径。
英文摘要：
We present SAGA, a versatile and adaptive framework for visuomotor control that can generalize across various environments, task objectives, and user specifications. To efficiently learn such capability, our key idea is to disentangle high-level semantic intent from low-level visuomotor control by explicitly grounding task objectives in the observed environment. Using an affordance-based task representation, we express diverse and complex behaviors in a unified, structured form. By leveraging multimodal foundation models, SAGA grounds the proposed task representation to the robot's visual observation as 3D affordance heatmaps, highlighting task-relevant entities while abstracting away spurious appearance variations that would hinder generalization. These grounded affordances enable us to effectively train a conditional policy on multi-task demonstration data for whole-body control. In a unified framework, SAGA can solve tasks specified in different forms, including language instructions, selected points, and example demonstrations, enabling both zero-shot execution and few-shot adaptation. We instantiate SAGA on a quadrupedal manipulator and conduct extensive experiments across eleven real-world tasks. SAGA consistently outperforms end-to-end and modular baselines by substantial margins. Together, these results demonstrate that structured affordance grounding offers a scalable and effective pathway toward generalist mobile manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
