arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.18873cs.ROcs.CV
收藏

LoopWAM:基于部署 rollouts 的闭环通用策略改进

LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts

Tsinghua University(清华大学);Dexmal;Loopeva

Wenxuan Guo, Yingping Liang, Zhaoguo Wang, Ziyuan Li, Angyuan Ma, Zhenyu Wu, Chengkun Wang, Bingyao Yu, Ze Chen, Borui Zhang, Xiuwei Xu, Meng Zhang, Erjin Zhou, Jiwen Lu

AI总结:

LoopWAM 是一种闭环世界-动作模型,利用自身及其他部署策略的混合质量 rollouts 学习,在多个机器人基准测试中实现了显著的策略改进,取得了最优结果。

摘要:

仅扩展专家演示无法让通用机器人策略从部署中实现改进。真实世界的 rollouts 包含成功、次优行为和失败:模仿每一个记录的动作会强化不良行为,而过滤掉失败则会丢弃物理交互及其后果的证据。我们引入 LoopWAM,一种闭环世界-动作模型,它直接从这种全质量范围学习,使用来自自身和其他已部署策略的 rollouts。依赖源的噪声采样将行为学习和动作条件世界建模统一起来,而质量引导和质量加权监督则保留了跨所有质量的完整视频监督并调节动作模仿。视频和动作标记在并行去噪过程中全程交互,使学习到的后果能够直接为动作生成提供信息。在 RoboChallenge Table30-V2 上,三轮部署-训练将成功率从 23.67% 提升至 43.33%,增幅为 19.7 个百分点,最终得分为 56.80。LoopWAM 在 RoboTwin 2.0 上还实现了 94.5% 的平均成功率,在 RoboCasa365 上实现了 49.5% 的整体成功率,且无需额外的具身预训练,在所有三个基准测试中均取得了对比方法中的最佳整体结果。在具有挑战性的叠衬衫任务上,单任务后训练后的两轮部署驱动改进将成功率从 13.20% 提升至 55.56%。这些结果证明了在多任务学习和单任务后训练两种场景下,基于混合质量经验的闭环改进是可行的。项目页面:this https URL

英文摘要:

Scaling expert demonstrations alone does not enable generalist robot policies to improve from deployment. Real-world rollouts contain successes, suboptimal behavior, and failures: imitating every recorded action can reinforce poor behavior, while filtering out failures discards evidence of physical interactions and their consequences. We introduce LoopWAM, a closed-loop world-action model that learns directly from this full quality spectrum, using rollouts from both itself and other deployed policies. Source-dependent noise sampling unifies behavior learning and action-conditioned world modeling, while quality steering and quality-weighted supervision retain full video supervision across qualities and regulate action imitation. Video and action tokens interact throughout parallel denoising, allowing learned consequences to inform action generation directly. On RoboChallenge Table30-V2, three deployment-and-training rounds raise success from 23.67% to 43.33%, a 19.7-percentage-point gain, with a final score of 56.80. LoopWAM also achieves 94.5% average success on RoboTwin 2.0 and 49.5% overall success on RoboCasa365 without additional embodied pretraining, obtaining the best overall results among the compared methods across all three benchmarks. On challenging shirt folding task, two rounds of deployment-driven improvement after single-task post-training raise success from 13.20% to 55.56%. These results demonstrate closed-loop improvement from mixed-quality experience under both multi-task learning and single-task post-training. Project page: https://loop-wam.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑