端到端驾驶数据集的隐藏偏差
Hidden Biases of End-to-End Driving Datasets
AI总结:
该研究针对端到端驾驶数据集影响被忽视的问题,系统分析训练数据集得出三点关键见解,据此优化的模型在2024年CARLA挑战赛获佳绩并刷新Bench2Drive基准,还指出评估指标缺陷并提出修改方案。
摘要:
端到端驾驶系统已取得快速进展,但目前尚未应用于颇具挑战性的全新CARLA Leaderboard 2.0(CARLA排行榜2.0)。此外,尽管已有大量关于端到端架构与训练策略的文献,但训练数据集的影响往往被忽视。在本研究中,我们首次尝试面向Leaderboard 2.0开展端到端驾驶研究。我们并未研究架构,而是系统分析了训练数据集,得出了全新见解:(1)专家风格会显著影响下游策略性能。(2)在复杂数据集中,不应基于类别频率这类简单标准对帧加权。(3)相反,估计某一帧相较于前序帧是否改变目标标签,可在不丢失重要信息的前提下缩小数据集规模。通过整合这些发现,我们的模型在2024年CARLA挑战赛的地图赛道和传感器赛道中分获第一名和第二名,并在Bench2Drive测试路线上创下了新的最优性能(state-of-the-art)。最后,我们发现了当前评估指标中的一处设计缺陷,并为未来的挑战赛提出了修改方案。我们的数据集、代码和预训练模型已在https://github.com/autonomousvision/carla_garage公开。
英文摘要:
End-to-end driving systems have made rapid progress, but have so far not been applied to the challenging new CARLA Leaderboard 2.0. Further, while there is a large body of literature on end-to-end architectures and training strategies, the impact of the training dataset is often overlooked. In this work, we make a first attempt at end-to-end driving for Leaderboard 2.0. Instead of investigating architectures, we systematically analyze the training dataset, leading to new insights: (1) Expert style significantly affects downstream policy performance. (2) In complex data sets, the frames should not be weighted on the basis of simplistic criteria such as class frequencies. (3) Instead, estimating whether a frame changes the target labels compared to previous frames can reduce the size of the dataset without removing important information. By incorporating these findings, our model ranks first and second respectively on the map and sensors tracks of the 2024 CARLA Challenge, and sets a new state-of-the-art on the Bench2Drive test routes. Finally, we uncover a design flaw in the current evaluation metrics and propose a modification for future challenges. Our dataset, code, and pre-trained models are publicly available at https://github.com/autonomousvision/carla_garage.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。