arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.08932cs.AIcs.RO
收藏

面向动态环境中自主决策的深度注意力驱动强化学习(DAD-RL)

Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment

Indian Institute of Science(印度科学学院);Indian Institute of Science Education and Research, Bhopal(博帕尔印度科学教育与研究学院);Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science(印度科学学院罗伯特·博世网络物理系统中心)

Jayabrata Chowdhury, Venkataramanan Shivaraman, Sumit Dangi, Suresh Sundaram, P. B. Sujit

AI总结:

针对城市自动驾驶决策中Transformer架构计算复杂的问题,提出DAD-RL框架,通过时空注意力编码和上下文编码动态融合周围车辆重要性,并在SMARTS基准上验证其优于现有方法。

摘要:

城市环境中自动驾驶车辆(AV)的决策本质上具有挑战性,因为需要与周围车辆进行动态交互。为了安全规划,AV必须理解场景中各种时空交互的权重。当前的研究主要使用庞大的Transformer架构来编码交互,主要用于轨迹预测,导致计算复杂度增加。为了解决这一问题,同时不牺牲时空理解和性能,我们提出了简单的深度注意力驱动强化学习(DADRL)框架,该框架动态分配并将周围车辆的重要性纳入自车的强化学习驱动决策过程中。我们引入了一种以AV为中心的时空注意力编码(STAE)机制,用于学习与不同周围车辆的动态交互。为了理解地图和路线上下文,我们采用上下文编码器从上下文地图中提取特征。时空表示与上下文编码相结合,提供了全面的状态表示。所得模型使用Soft Actor Critic(SAC)算法进行训练。我们在无交通信号的SMARTS城市基准场景上评估了所提出的框架,证明DADRL优于近期最先进的方法。此外,消融研究强调了上下文编码器和时空注意力编码器在实现卓越性能中的重要性。

英文摘要:

Autonomous Vehicle (AV) decision making in urban environments is inherently challenging due to the dynamic interactions with surrounding vehicles. For safe planning, AV must understand the weightage of various spatiotemporal interactions in a scene. Contemporary works use colossal transformer architectures to encode interactions mainly for trajectory prediction, resulting in increased computational complexity. To address this issue without compromising spatiotemporal understanding and performance, we propose the simple Deep Attention Driven Reinforcement Learning (DADRL) framework, which dynamically assigns and incorporates the significance of surrounding vehicles into the ego's RL driven decision making process. We introduce an AV centric spatiotemporal attention encoding (STAE) mechanism for learning the dynamic interactions with different surrounding vehicles. To understand map and route context, we employ a context encoder to extract features from context maps. The spatiotemporal representations combined with contextual encoding provide a comprehensive state representation. The resulting model is trained using the Soft Actor Critic (SAC) algorithm. We evaluate the proposed framework on the SMARTS urban benchmarking scenarios without traffic signals to demonstrate that DADRL outperforms recent state of the art methods. Furthermore, an ablation study underscores the importance of the context-encoder and spatio temporal attention encoder in achieving superior performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑