用于机器人控制的智能体AI:灵活但依然脆弱
Agentic AI for Robot Control: Flexible but still Fragile
AI总结:
本文提出一种基于推理型语言模型的智能体控制系统,通过迭代规划-执行循环选择并调用机器人技能完成任务,在两种物理平台上实验表明系统灵活但存在非确定性行为、指令遵循错误和提示敏感等脆弱性。
摘要:
最近的工作利用生成模型的能力和常识先验进行机器人控制。在本文中,我们提出了一种智能体控制系统,其中具有推理能力的语言模型通过迭代规划器和执行器循环选择并调用机器人技能来规划和执行任务。我们在两种物理机器人平台上部署该系统,分别用于:(i) 室内移动操作(Mobipick)中的桌面抓取、放置和箱子插入,以及 (ii) 自主农业导航和感知(Valdemar)。两种设置都涉及不确定性、部分可观测性、传感器噪声和模糊的自然语言命令。该系统公开了其规划和决策过程的结构化内省,通过显式事件检查对外部事件做出反应,并支持操作员干预以修改或重定向正在进行的执行。在两个平台上,我们的概念验证实验揭示了显著的脆弱性,包括非确定性次优行为、指令遵循错误以及对提示规范的高度敏感性。同时,该架构是灵活的:转移到不同的机器人和任务领域主要需要更新系统提示(领域模型、可供性和动作目录)并将相同的工具接口重新绑定到平台特定的技能API。
英文摘要:
Recent work leverages the capabilities and commonsense priors of generative models for robot control. In this paper, we present an agentic control system in which a reasoning-capable language model plans and executes tasks by selecting and invoking robot skills within an iterative planner and executor loop. We deploy the system on two physical robot platforms in two settings: (i) tabletop grasping, placement, and box insertion in indoor mobile manipulation (Mobipick) and (ii) autonomous agricultural navigation and sensing (Valdemar). Both settings involve uncertainty, partial observability, sensor noise, and ambiguous natural-language commands. The system exposes structured introspection of its planning and decision process, reacts to exogenous events via explicit event checks, and supports operator interventions that modify or redirect ongoing execution. Across both platforms, our proof-of-concept experiments reveal substantial fragility, including non-deterministic suboptimal behavior, instruction-following errors, and high sensitivity to prompt specification. At the same time, the architecture is flexible: transfer to a different robot and task domain largely required updating the system prompt (domain model, affordances, and action catalogue) and re-binding the same tool interface to the platform-specific skill API.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。