RACER：用于模仿学习的丰富语言引导失败恢复策略
RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning
AI总结：
针对机器人操作缺乏自我恢复机制及简单语言指令局限的问题，提出RACER框架，结合失败恢复数据与丰富语言描述，利用VLM在线监督纠正错误，在多项任务中显著提升了机器人的控制性能。
摘要：
由于缺乏从失败中自我恢复的机制，以及简单语言指令在引导机器人动作方面的局限性，为机器人操作开发鲁棒且可纠正的视觉运动策略具有挑战性。为了解决这些问题，我们提出了一种可扩展的数据生成流水线，该流水线自动用失败恢复轨迹和细粒度语言注释来增强专家演示以用于训练。然后我们引入了 Rich languAge-guided failure reCovERy (RACER)，这是一个 supervisor-actor 框架，它将失败恢复数据与丰富的语言描述相结合以增强机器人控制。RACER 的特点是包含一个视觉语言模型 (VLM) 作为在线监督器，为错误纠正和任务执行提供详细的语言指导，以及一个语言条件的视觉运动策略作为执行器来预测下一个动作。我们的实验结果表明，RACER 在 RLbench 上的各种评估设置中均优于最先进的 Robotic View Transformer (RVT)，包括标准长时程任务、动态目标变更任务和零样本未见任务，在模拟和真实世界环境中均实现了卓越的性能。视频和代码可在：https://rich-language-failure-recovery.github.io 获取。
英文摘要：
Developing robust and correctable visuomotor policies for robotic manipulation is challenging due to the lack of self-recovery mechanisms from failures and the limitations of simple language instructions in guiding robot actions. To address these issues, we propose a scalable data generation pipeline that automatically augments expert demonstrations with failure recovery trajectories and fine-grained language annotations for training. We then introduce Rich languAge-guided failure reCovERy (RACER), a supervisor-actor framework, which combines failure recovery data with rich language descriptions to enhance robot control. RACER features a vision-language model (VLM) that acts as an online supervisor, providing detailed language guidance for error correction and task execution, and a language-conditioned visuomotor policy as an actor to predict the next actions. Our experimental results show that RACER outperforms the state-of-the-art Robotic View Transformer (RVT) on RLbench across various evaluation settings, including standard long-horizon tasks, dynamic goal-change tasks and zero-shot unseen tasks, achieving superior performance in both simulated and real world environments. Videos and code are available at: https://rich-language-failure-recovery.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
