从基础到操作：具身机器人系统中基础模型集成的案例研究
From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems
AI总结：
本文研究三种基础模型集成范式在机器人系统中的应用，通过复杂指令基础和物体操作任务的案例研究，揭示了零样本和少样本设置下泛化与数据效率的权衡，为语言驱动物理智能体的设计提供启示。
摘要：
基础模型（FMs）越来越多地用于在具身智能体中桥接语言和动作，但不同FM集成策略的操作特性仍未得到充分探索——特别是在变化环境中进行复杂指令跟随和多功能动作生成方面。本文研究了构建机器人系统的三种范式：隐式集成感知和规划的端到端视觉-语言-动作（VLA）模型，以及结合视觉-语言模型（VLMs）或多模态大语言模型（LLMs）的模块化流水线。我们通过两个聚焦的案例研究评估这些范式：一个评估细粒度指令理解和跨模态消歧的复杂指令基础任务，以及一个通过VLA微调实现技能迁移的物体操作任务。我们在零样本和少样本设置下的实验揭示了泛化和数据效率方面的权衡。通过探索性能极限，我们提炼了开发语言驱动物理智能体的设计启示，并概述了FM驱动机器人在真实世界条件下面临的新兴挑战和机遇。
英文摘要：
Foundation models (FMs) are increasingly used to bridge language and action in embodied agents, yet the operational characteristics of different FM integration strategies remain under-explored -- particularly for complex instruction following and versatile action generation in changing environments. This paper examines three paradigms for building robotic systems: end-to-end vision-language-action (VLA) models that implicitly integrate perception and planning, and modular pipelines incorporating either vision-language models (VLMs) or multimodal large language models (LLMs). We evaluate these paradigms through two focused case studies: a complex instruction grounding task assessing fine-grained instruction understanding and cross-modal disambiguation, and an object manipulation task targeting skill transfer via VLA finetuning. Our experiments in zero-shot and few-shot settings reveal trade-offs in generalization and data efficiency. By exploring performance limits, we distill design implications for developing language-driven physical agents and outline emerging challenges and opportunities for FM-powered robotics in real-world conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
