arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2503.21969cs.ROcs.AI

具身长时程操作:基于闭环代码生成与增量式少样本自适应

Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback

Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)

Yuan Meng, Xiangtong Yao, Haihui Ye, Yirui Zhou, Shengqiang Zhang, Zhenguo Sun, Xukun Li, Zhenshan Bing, Alois Knoll

AI总结:

本文提出一种闭环代码生成框架,舍弃预训练低层策略,利用LLM以CoT引导的增量少样本学习直接生成可执行代码规划,并通过RGB-D反馈实现恢复与重规划,在多个基准上取得最优性能。

摘要:

具身长时程操作要求机器人系统处理多模态输入(如视觉和自然语言),并将其转化为可执行动作。然而,现有的基于学习的方法通常依赖大规模、任务特定的数据集,难以泛化到未见场景。近期方法探索使用大语言模型(LLMs)作为高层规划器,利用自然语言将任务分解为子任务,并引导预训练的低层控制器。然而,这些方法假设低层策略能够完美执行,这在实际环境中存在噪声或次优行为时并不现实。为克服这一问题,我们完全舍弃预训练的低层策略,转而使用LLM在闭环框架内直接生成可执行代码规划。我们的规划器采用链式思维(CoT)引导的少样本学习,并利用增量式结构化的示例来生成鲁棒且可泛化的任务规划。与之互补,一个报告器利用RGB-D评估执行结果并提供结构化反馈,从而在部分可观测条件下实现失准恢复与重新规划。该设计消除了逐步推理,降低了计算开销,并限制了先前方法中观察到的误差累积。我们的框架在LoHoRavens、CALVIN、Franka Kitchen以及杂乱真实环境中的30多个已见和未见长时程任务上取得了最先进的性能。

英文摘要:

Robotic long-horizon manipulation requires robots to compose perception, reasoning, and action over extended task sequences, yet existing language-conditioned frameworks often rely on dense step-wise feedback, learned action policies, or unstructured prompting, which limits robustness and generalization. We propose DAHLIA, a data-agnostic code-generation framework that treats long-horizon manipulation as episodic task planning and evaluation. DAHLIA uses progressively organized in-context examples with chain-of-thought reasoning to guide an LLM in synthesizing executable code plans from language instructions. To improve robustness under partial observability, a VLM-based reporter evaluates task outcomes only after each execution episode and provides structured feedback for replanning, avoiding the overhead of per-step inference. Experiments on LoHoRavens, CALVIN, Franka Kitchen, and real-world manipulation tasks show that DAHLIA achieves strong performance on over 30 long-horizon tasks and improves generalization to unseen scenarios, including cluttered scenes and occluded objects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑