arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.24121cs.RO
收藏

在线世界建模实现真实世界逆强化学习从观察中学习

Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation

University of Washington(华盛顿大学)

Tyler Han, Bat Nemekhbold, Siyang Shen, Rohan Baijal, Richard Ebock, Harine Ravichandiran, Sanghun Jung, Kevin Huang, Byron Boots

AI总结:

提出MPAIL2方法,通过在线世界建模实现从观察中逆强化学习,首次在真实世界从零学习视觉操作任务,40分钟内达到82%成功率。

摘要:

当前机器人学习方法从根本上受限于一个或多个瓶颈:手工设计的奖励、仿真建模或动作监督(如遥操作),每个都需要大量的领域专业知识、工程工作和机器人操作员劳动。为了消除这些瓶颈,本工作通过从观察中逆强化学习(IRLfO)追求观察学习,其中仅假设可以访问任务观察(例如视频)。由于具有挑战性的设置和RL方法的局限性,IRLfO迄今为止对于真实世界机器人学习仍不实用。在这里,我们提出了第一个从零开始在真实世界中学习视觉操作的IRL方法,以及第一个跨视觉操作任务从零开始进行正向在线迁移的真实世界演示。在不到40分钟的时间内,MPAIL2从零开始学习拾取和放置任务,成功率达到82%,而具有相同交互和演示预算的RL和BC尽管有奖励和动作监督,仅达到0%和12%。交互式项目页面及训练视频:此 https URL

英文摘要:

Current methods in robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and robot-operator labor. Towards reducing these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g. video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82%, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision. Interactive project page with training videos: https://uwrobotlearning.github.io/mpail2/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑