挖掘长尾：面向自主运动规划中鲁棒离线强化学习的以数据为中心的关键性度量比较研究
Mining the Long Tail: A Comparative Study of Data-Centric Criticality Metrics for Robust Offline Reinforcement Learning in Autonomous Motion Planning
AI总结：
本研究针对自动驾驶离线强化学习中的长尾数据不平衡问题，系统比较了六种关键性加权数据筛选策略，发现基于模型不确定性的非均匀采样能将碰撞率降至5.5%，并揭示了时间步与场景级加权在安全性与规划上的权衡。
摘要：
离线强化学习（RL）为从大规模真实世界驾驶日志中训练自动驾驶车辆（AV）规划策略提供了一种有前景的范式。然而，这些日志中存在极端的数据不平衡，普通场景的数量远远超过罕见的“长尾”事件，这导致在使用标准均匀数据采样时，策略变得脆弱且不安全。在本研究中，我们通过一项系统的大规模比较研究来应对这一挑战，该研究旨在比较旨在使学习过程聚焦于信息丰富样本的数据筛选策略。我们研究了六种不同的关键性加权方案，它们被分为三类：基于启发式的、基于不确定性的和基于行为的。这些方案在两个时间尺度上进行评估，即单个时间步长和完整场景。我们训练了七个目标条件的保守Q学习（CQL）智能体，采用了基于注意力机制的先进架构，并在高保真Waymax模拟器中对其进行评估。结果表明，所有数据筛选方法均显著优于基线。值得注意的是，使用模型不确定性作为信号的数据驱动筛选实现了最显著的安全性提升，将碰撞率降低了近三倍（从16.0%降至5.5%）。此外，我们发现了一个明确的权衡：时间步长级别的加权在反应式安全性方面表现优异，而场景级别的加权则能改善长时程规划。我们的工作为离线RL中的数据筛选提供了一个全面的框架，并强调智能的非均匀采样是构建安全可靠自主智能体的关键组成部分。
英文摘要：
Offline Reinforcement Learning (RL) presents a promising paradigm for training autonomous vehicle (AV) planning policies from large-scale, real-world driving logs. However, the extreme data imbalance in these logs, where mundane scenarios vastly outnumber rare "long-tail" events, leads to brittle and unsafe policies when using standard uniform data sampling. In this work, we address this challenge through a systematic, large-scale comparative study of data curation strategies designed to focus the learning process on information-rich samples. We investigate six distinct criticality weighting schemes which are categorized into three families: heuristic-based, uncertainty-based, and behavior-based. These are evaluated at two temporal scales, the individual timestep and the complete scenario. We train seven goal-conditioned Conservative Q-Learning (CQL) agents with a state-of-the-art, attention-based architecture and evaluate them in the high-fidelity Waymax simulator. Our results demonstrate that all data curation methods significantly outperform the baseline. Notably, data-driven curation using model uncertainty as a signal achieves the most significant safety improvements, reducing the collision rate by nearly three-fold (from 16.0% to 5.5%). Furthermore, we identify a clear trade-off where timestep-level weighting excels at reactive safety while scenario-level weighting improves long-horizon planning. Our work provides a comprehensive framework for data curation in Offline RL and underscores that intelligent, non-uniform sampling is a critical component for building safe and reliable autonomous agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
