arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07837cs.RO
收藏

RLinf-USER: 一个统一且可扩展的系统,用于具身人工智能中的现实世界在线策略学习

USER: A Unified and Extensible System for Online Real-World Policy Learning in Embodied AI

Hongzhi Zang, Shu'ang Yu, Hao Lin, Tianxing Zhou, Zefang Huang, Zhen Guo, Xin Xu, Yuze Sheng, Jiakai Zhou, Shizhe Zhang, Feng Gao, Wenhao Tang, Yufeng Yue, Quan… 展开作者

Hongzhi Zang, Shu'ang Yu, Hao Lin, Tianxing Zhou, Zefang Huang, Zhen Guo, Xin Xu, Yuze Sheng, Jiakai Zhou, Shizhe Zhang, Feng Gao, Wenhao Tang, Yufeng Yue, Quanlu Zhang, Xinlei Chen, Chao Yu, Yu Wang

AI总结:

RLinf-USER是一个统一且可扩展的系统,用于现实世界中的在线策略学习,通过统一硬件抽象层和异步框架实现多机器人协调与长时训练。

摘要:

在线策略学习直接在物理世界中进行,是具身智能的一个有前景但具有挑战性的方向。与仿真不同,现实世界系统无法任意加速、廉价重置或大规模复制,这使得可扩展的数据收集、异构部署和长时间有效训练困难。这些挑战表明,现实世界策略学习不仅是一个算法问题,从根本上说是一个系统问题。我们提出了USER,一个统一且可扩展的系统,用于现实世界的在线策略学习。USER通过统一的硬件抽象层将物理机器人作为第一类硬件资源与GPU并列,从而实现异构机器人的自动发现、管理和调度。为了解决云边通信问题,USER引入了基于隧道的通信平面、分布式数据通道用于流量定位,以及流式多处理器感知的权重同步,以调节GPU侧的开销。在此基础设施之上,USER将学习组织为一个完全异步的框架,配备持久、缓存感知的缓冲区,从而能够高效地进行长时间实验,具有强大的崩溃恢复能力和历史数据的重用。此外,USER提供了可扩展的抽象,用于奖励、算法和策略,支持在统一管道中在线模仿或强化学习CNN/MLP、生成策略以及大规模视觉-语言-动作(VLA)模型。在仿真和现实世界中的结果表明,USER能够实现多机器人协调、异构机械臂、与大模型的云边协作以及长时间的异步训练,提供了一个统一且可扩展的系统基础,用于现实世界的在线策略学习。

英文摘要:

Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, suggesting that real-world policy learning is not merely an algorithmic problem, but inherently a systems problem. We present \user, a \underline{U}nified and extensible \underline{S}yst\underline{E}m for online real-world policy lea\underline{R}ning. On the systems side, \user introduces a hardware abstraction layer for unified robot management and an adaptive communication plane that enables efficient cloud-edge training. On the learning side, \user adopts a fully asynchronous training framework, designs a persistent and cache-aware replay buffer, and provides extensible abstractions for rewards, algorithms, and policies. Experiments in both simulation and the real world demonstrate that \user supports multi-robot coordination, heterogeneous manipulators, cloud-edge training with large models, and long-running asynchronous training. Together, these capabilities establish \user as a unified and extensible systems foundation for online real-world policy learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑