从模仿到优化：自动驾驶离线学习的比较研究
From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
AI总结：
本文比较行为克隆与保守Q学习在自动驾驶离线学习中的表现，提出基于Transformer的实体中心状态表示流程，并在Waymo数据集上证明CQL相比最强BC基线成功率提升3.2倍、碰撞率降低7.4倍。
摘要：
从大规模真实世界数据集中学习稳健的驾驶策略是自动驾驶领域的核心挑战，因为在线数据收集往往不安全且不切实际。虽然行为克隆（Behavioral Cloning, BC）提供了一种直接的模仿学习方法，但用BC训练的策略以脆弱著称，并在闭环执行中遭受复合误差的影响。本文提出了一个完整的流程和一项比较研究来解决这一局限。我们首先开发了一系列日益复杂的BC基线，最终得到一个基于Transformer的模型，该模型在结构化的、以实体为中心的状态表示上运行。虽然该模型实现了较低的模仿损失，但我们表明它在长时域仿真中仍然失败。随后我们证明，通过将最先进的离线强化学习算法——保守Q学习（Conservative Q-Learning, CQL）应用于相同的数据和架构，可以学习到显著更稳健的策略。利用精心设计的奖励函数，CQL智能体学习到一个保守的价值函数，使其能够从轻微错误中恢复并避免分布外状态。在Waymo Open Motion Dataset的1,000个未见场景上进行的大规模评估中，我们最终的CQL智能体相比最强的BC基线实现了3.2倍的成功率和7.4倍的碰撞率降低，证明离线强化学习方法对于从静态专家数据中学习稳健的长时域驾驶策略至关重要。
英文摘要：
Learning robust driving policies from large-scale, real-world datasets is a central challenge in autonomous driving, as online data collection is often unsafe and impractical. While Behavioral Cloning (BC) offers a straightforward approach to imitation learning, policies trained with BC are notoriously brittle and suffer from compounding errors in closed-loop execution. This work presents a comprehensive pipeline and a comparative study to address this limitation. We first develop a series of increasingly sophisticated BC baselines, culminating in a Transformer-based model that operates on a structured, entity-centric state representation. While this model achieves low imitation loss, we show that it still fails in long-horizon simulations. We then demonstrate that by applying a state-of-the-art Offline Reinforcement Learning algorithm, Conservative Q-Learning (CQL), to the same data and architecture, we can learn a significantly more robust policy. Using a carefully engineered reward function, the CQL agent learns a conservative value function that enables it to recover from minor errors and avoid out-of-distribution states. In a large-scale evaluation on 1,000 unseen scenarios from the Waymo Open Motion Dataset, our final CQL agent achieves a 3.2x higher success rate and a 7.4x lower collision rate than the strongest BC baseline, proving that an offline RL approach is critical for learning robust, long-horizon driving policies from static expert data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
