arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.08674cs.RO
收藏

注意你的驾驶方式:面向越野驾驶的安全自适应基于模型的强化学习

Pay Attention to How You Drive: Safe and Adaptive Model-Based Reinforcement Learning for Off-Road Driving

Carnegie Mellon University(卡内基梅隆大学)

Sean J. Wang, Honghao Zhu, Aaron M. Johnson

AI总结:

针对越野自动驾驶,提出基于模型的强化学习方法,利用系统识别变换器和自适应动力学模型平衡鲁棒性与适应性,并通过风险感知MPPI控制器实现安全控制,在仿真和真实环境中显著提升速度且保持安全。

摘要:

自主越野驾驶具有挑战性,因为机器人采取的危险行动可能导致灾难性损坏。因此,在仿真中开发控制器通常是可取的,因为它提供了更安全且更经济的替代方案。然而,由于非结构化环境中复杂的机器人动力学和地形交互,准确建模机器人动力学是困难的。域随机化通过随机化仿真动力学参数来解决这一问题,但这种方法以牺牲性能换取鲁棒性,导致策略对于任何目标动力学都是次优的。我们提出了一种新颖的基于模型的强化学习方法,旨在平衡鲁棒性与适应性。我们的方法在多种仿真动力学下训练一个系统识别变换器(SIT)和一个自适应动力学模型(ADM)。SIT利用注意力机制将目标系统的状态转移观测提炼为上下文向量,该向量为其目标动力学提供了抽象表示。基于此,ADM对系统动力学进行概率建模。在线阶段,我们使用风险感知模型预测路径积分控制器(MPPI)在机器人当前对动力学的理解下安全地控制机器人。我们在仿真以及多个真实世界环境中证明,该方法在初始化时能够实现更安全的行为,并随着对目标系统动力学理解的增加(通过更多观测)而变得不那么保守(即更快)。特别是,我们的方法在保持不同环境下安全性的同时,与非自适应基线相比,单圈时间大约提高了41%。

英文摘要:

Autonomous off-road driving is challenging as risky actions taken by the robot may lead to catastrophic damage. As such, developing controllers in simulation is often desirable as it provides a safer and more economical alternative. However, accurately modeling robot dynamics is difficult due to the complex robot dynamics and terrain interactions in unstructured environments. Domain randomization addresses this problem by randomizing simulation dynamics parameters, however this approach sacrifices performance for robustness leading to policies that are sub-optimal for any target dynamics. We introduce a novel model-based reinforcement learning approach that aims to balance robustness with adaptability. Our approach trains a System Identification Transformer (SIT) and an Adaptive Dynamics Model (ADM) under a variety of simulated dynamics. The SIT uses attention mechanisms to distill state-transition observations from the target system into a context vector, which provides an abstraction for its target dynamics. Conditioned on this, the ADM probabilistically models the system's dynamics. Online, we use a Risk-Aware Model Predictive Path Integral controller (MPPI) to safely control the robot under its current understanding of the dynamics. We demonstrate in simulation as well as in multiple real-world environments that this approach enables safer behaviors upon initialization and becomes less conservative (i.e. faster) as its understanding of the target system dynamics improves with more observations. In particular, our approach results in an approximately 41% improvement in lap-time over the non-adaptive baseline while remaining safe across different environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑