arXiv 2501.122962025-07-23cs.CVcs.AI

RALAD：利用检索增强学习弥合自动驾驶中的真实到仿真域差距

RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learning

Department of Future Science and Engineering, Soochow University（苏州大学未来科学与工程学院）；Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；Department of Computer Science, McGill University（麦吉尔大学计算机科学系）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）；Department of Computer Science, Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学计算机科学系）

Jiacheng Zuo, Haibo Hu, Zikang Zhou, Yufei Cui, Ziquan Liu, Jianping Wang, Nan Guan, Jin Wang, Chun Jason Xue

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出RALAD框架，通过增强OT域适应、统一架构和冻结高成本层的高效微调，低成本弥合自动驾驶真实到仿真差距，并在三种模型上显著提升仿真性能且保持真实场景精度。

摘要：

在追求鲁棒自动驾驶系统的过程中，基于真实世界数据集训练的模型往往难以适应新环境，尤其是在面对极端天气条件等边角案例时。在真实世界中收集这些边角案例并非易事，因此需要使用仿真器进行验证。然而，高昂的计算成本和数据分布中的域差距阻碍了真实与仿真驾驶场景之间的无缝转换。为应对这一挑战，我们提出了用于自动驾驶的检索增强学习（Retrieval-Augmented Learning for Autonomous Driving，RALAD），这是一种旨在以低成本弥合真实到仿真差距的新颖框架。RALAD具有三项主要设计，包括：（1）通过增强的最优传输（Optimal Transport，OT）方法进行域适应，该方法同时考虑单个图像距离和分组图像距离；（2）一个简单且统一的框架，可应用于各种模型；（3）高效的微调技术，在冻结计算成本高昂的层的同时保持鲁棒性。实验结果表明，RALAD在三种不同模型上弥补了仿真环境中的性能下降，同时保持了真实世界场景中的准确性。以Cross View为例，在RALAD微调前后，真实世界场景中的mIOU和mAP指标保持稳定，而在仿真环境中，mIOU和mAP指标分别提升了10.30%和12.29%。此外，我们方法的重新训练成本降低了约88.1%。我们的代码可在https://github.com/JiachengZuo/RALAD.git获取。

英文摘要：

In the pursuit of robust autonomous driving systems, models trained on real-world datasets often struggle to adapt to new environments, particularly when confronted with corner cases such as extreme weather conditions. Collecting these corner cases in the real world is non-trivial, which necessitates the use of simulators for validation. However,the high computational cost and the domain gap in data distribution have hindered the seamless transition between real and simulated driving scenarios. To tackle this challenge, we propose Retrieval-Augmented Learning for Autonomous Driving (RALAD), a novel framework designed to bridge the real-to-sim gap at a low cost. RALAD features three primary designs, including (1) domain adaptation via an enhanced Optimal Transport (OT) method that accounts for both individual and grouped image distances, (2) a simple and unified framework that can be applied to various models, and (3) efficient fine-tuning techniques that freeze the computationally expensive layers while maintaining robustness. Experimental results demonstrate that RALAD compensates for the performance degradation in simulated environments while maintaining accuracy in real-world scenarios across three different models. Taking Cross View as an example, the mIOU and mAP metrics in real-world scenarios remain stable before and after RALAD fine-tuning, while in simulated environments,the mIOU and mAP metrics are improved by 10.30% and 12.29%, respectively. Moreover, the re-training cost of our approach is reduced by approximately 88.1%. Our code is available at https://github.com/JiachengZuo/RALAD.git.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。