MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving
School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）；State Key Laboratory of Intelligent Transportation System, Beihang University（北京航空航天大学智能交通系统国家重点实验室）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新院）；Contemporary Amperex Technology Co., Limited (CATL)（当代电动车技术有限公司（CATL））；Research Institute of Aero-Engine, Beihang University（北京航空航天大学航空发动机研究院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；China Automotive Engineering Research Institute Co., Ltd.（中国汽车工程研究院股份有限公司）
Bin Sun, Yaoguang Cao, Yan Wang, Rui Wang, Jiachen Shang, Xiejie Feng, Jiayi Lu, Jia Shi, Shichun Yang, Xiaoyu Yan, Ziying Song
End-to-End autonomous driving (E2E-AD) has emerged as a new paradigm, where trajectory planning plays a crucial role. Existing studies mainly follow two directions: trajectory generation oriented, which focuses on producing high-quality trajectories with simple decision mechanisms, and trajectory selection oriented, which performs multi-dimensional evaluation to select the best trajectory yet lacks sufficient generative capability. In this work, we propose MindDrive, a harmonized framework that integrates high-quality trajectory generation with comprehensive decision reasoning. It establishes a structured reasoning paradigm of "context simulation - candidate generation - multi-objective trade-off". In particular, the proposed Future-aware Trajectory Generator (FaTG), based on a World Action Model (WaM), performs ego-conditioned "what-if" simulations to predict potential future scenes and generate foresighted trajectory candidates. Building upon this, the VLM-oriented Evaluator (VLoE) leverages the reasoning capability of a large vision-language model to conduct multi-objective evaluations across safety, comfort, and efficiency dimensions, leading to reasoned and human-aligned decision making. Extensive experiments on the NAVSIM-v1 and NAVSIM-v2 benchmarks demonstrate that MindDrive achieves state-of-the-art performance across multi-dimensional driving metrics, significantly enhancing safety, compliance, and generalization. This work provides a promising path toward interpretable and cognitively guided autonomous driving.
MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving
Amap, Alibaba Group（阿里集团蚂巴公司）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The Chinese University of Hong Kong（香港中文大学）；Xi’an Jiaotong University（西安交通大学）
3D object detection is an essential task in autonomous driving. Recent techniques excel with highly accurate detection rates, provided the 3D input data is obtained from precise but expensive LiDAR technology. Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies --- a gap that is commonly attributed to poor image-based depth estimation. However, in this paper we argue that it is not the quality of the data but its representation that accounts for the majority of the difference. Taking the inner workings of convolutional neural n…
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domain. However, for high-stakes applications (e.g. autonomous driving), it is common to have a modest amount of human-labeled real data in addition to plentiful auto-labeled source data (e.g. from a driving simulator). We study this setting of supervised sim2real DA applied to 2D object detection. We propose Domain Translation via Conditional Alignment and Reweighting (CARE) a novel algorithm that systematically exploits…
RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos
Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Environmental perception with the multi-modal fusion of radar and camera is crucial in autonomous driving to increase accuracy, completeness, and robustness. This paper focuses on utilizing millimeter-wave (MMW) radar and camera sensor fusion for 3D object detection. A novel method that realizes the feature-level fusion under the bird's-eye view (BEV) for a better feature representation is proposed. Firstly, radar points are augmented with temporal accumulation and sent to a spatial-temporal encoder for radar feature extraction. Meanwhile, multi-scale image 2D features which adapt to various s…
Robert Bosch Corporate Research（博世企业研究院）；KLA（科磊）；Birla Institute of Technology and Science, Pilani（皮拉尼比特拉技术与科学学院）
While developing perception based deep learning models, the benefit of synthetic data is enormous. However, performance of networks trained with synthetic data for certain computer vision tasks degrade significantly when tested on real world data due to the domain gap between them. One of the popular solutions in bridging this gap between synthetic and actual world data is to frame it as a domain adaptation task. In this paper, we propose and evaluate novel ways for the betterment of such approaches. In particular we build upon the method of UNIT-GAN. In normal GAN training for the task of dom…
RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learning
Department of Future Science and Engineering, Soochow University（苏州大学未来科学与工程学院）；Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；Department of Computer Science, McGill University（麦吉尔大学计算机科学系）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）；Department of Computer Science, Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学计算机科学系）
LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers
Institute for Intelligent Systems, Esslingen University of Applied Sciences（智能系统研究所，埃斯林根应用科学大学）；Department of Computer Science, University of Freiburg（计算机科学系，弗赖堡大学）
CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Li Auto Inc.（利汽车公司）
Detector Collapse: Physical-World Backdooring Object Detection to Catastrophic Overload or Blindness in Autonomous Driving
School of Cyber Science and Engineering, Huazhong University of Science and Technology（华中科技大学网络空间科学与工程学院）；Hubei Engineering Research Center on Big Data Security（湖北省大数据安全工程研究中心）；Hubei Key Laboratory of Distributed System Security（分布式系统安全湖北省重点实验室）；National Engineering Research Center for Big Data Technology and System（国家大数据技术与系统工程研究中心）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）；University of California, Berkeley（加州大学伯克利分校）
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与出行学院）；DiDi Global（滴滴全球公司）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）
Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Training
Institute of Trustworthy Embodied AI, Fudan University（可信具身人工智能研究院，复旦大学）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；The Hong Kong Polytechnic University（香港理工大学）；NVIDIA
Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms
School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；Hong Kong Generative AI Research and Development Center（香港生成式人工智能研究与开发中心）
Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心，湖南大学）；Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（人机学与机器人研究所，卡尔斯鲁厄技术大学）
