arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02508cs.ROcs.AIcs.LG
收藏

面向真实世界驾驶的物理信息模仿强化学习

Physics-informed Imitative Reinforcement Learning for Real-world Driving

Robotics and Autonomous Systems Thrust, Systems Hub, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)机器人与自主系统 thrust、系统枢纽);Department of Computer Science and Engineering, School of Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系、工程学院)

Hang Zhou, Yihao Qin, Dan Xu, Yiding Ji

AI总结:

针对模仿强化学习在动态闭环环境中因优化目标冲突和样本低效导致性能下降的问题,提出完全数据驱动的物理信息IRL方法,利用联合优化让车辆动力学物理规律自然涌现,在Waymax基准上显著降低了碰撞率和驶出道路率。

摘要:

模仿强化学习(IRL)的最新进展显著增强了自主智能体吸收专家演示的能力,使其在一系列高难度任务中能够快速获取技能。然而,当这类基于学习的智能体向高度动态的闭环环境迁移知识时,面临着重大挑战。其性能受到模仿学习(IL)与强化学习(RL)之间相互冲突的优化目标、样本低效,以及揭示隐式世界模型与物理规律的复杂性的显著影响。为了应对这一挑战,我们提出了一种完全数据驱动的物理信息IRL方法。该方法利用专家演示数据和探索性数据,并采用联合优化目标,使得车辆动力学的底层物理原理能够从训练过程中自然涌现。通过实证实验对性能进行了评估,在Waymax基准的闭环设置下,结果超越了流行的IL、RL和IRL算法。与基线方法相比,我们的方法碰撞率降低了37.8%,驶出道路率降低了22.2%。

英文摘要:

Recent advances in imitative reinforcement learning (IRL) have considerably enhanced the ability of autonomous agents to assimilate expert demonstrations, leading to rapid skill acquisition in a range of demanding tasks. However, such learning-based agents face significant challenges when transferring knowledge to highly dynamic closed-loop environments. Their performance is significantly impacted by the conflicting optimization objectives of imitation learning (IL) and reinforcement learning (RL), sample inefficiency, and the complexity of uncovering the hidden world model and physics. To address this challenge, we propose a physics-informed IRL that is entirely data-driven. It leverages both expert demonstration data and exploratory data with a joint optimization objective, allowing the underlying physical principles of vehicle dynamics to emerge naturally from the training process. The performance is evaluated through empirical experiments and results exceed popular IL, RL and IRL algorithms in closed-loop settings on Waymax benchmark. Our approach exhibits 37.8% reduction in collision rate and 22.2% reduction in off-road rate compared to the baseline method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑