For end-to-end autonomous driving (E2E-AD), the evaluation system remains an open problem. Existing closed-loop evaluation protocols usually rely on simulators like CARLA being less realistic; while NAVSIM using real-world vision data, yet is limited to fixed planning trajectories in short horizon and assumes other agents are not reactive.
We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation. Unlike existing video generative models for AD, the proposed designs are tailored for interactive simulation, where sensor rendering and behavior rollout are decoupled by applying a separate behavioral controller to simulate the reactions of surrounding agents. As a result, the renderer could focus on image fidelity, control adherence, and spatial-temporal coherence. For temporal consistency, due to the step-wise interaction nature of simulation, we design a noise modulating temporal encoder with Gaussian blurring to encourage long-horizon autoregressive rollout of image sequences without deteriorating distribution shifts. For spatial consistency, a retrieval mechanism, which takes the spatially nearest images as references, is introduced to to ensure scene-level rendering fidelity during the generation process. The spatial relations between target and reference are explicitly modeled with 3D relative position encodings and the potential over-reliance of reference images is mitigated with hierarchical sampling and classifier-free guidance.
We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance. We further integrate Bench2Drive-R into nuPlan and evaluate the generative qualities with closed-loop simulation results. We will open source our code.
Autonomous Driving at Intersections: A Critical-Turning-Point Approach for Left Turns
University of Waterloo(滑铁卢大学);Waytous Inc.(Waytous公司);Autonomous Driving Lab, Tencent(腾讯自动驾驶实验室);Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University(清华大学自动化系清华大学信息科学与技术国家实验室)
ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Driving
School of Computer Science & School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学计算机科学与技术学院、人工智能学院);Great Wall Motor(长城汽车);Institute of Trustworthy Embodied AI (TEAI), Fudan University(复旦大学可信具身人工智能研究所);School of Computer Science, Wuhan University(武汉大学计算机学院);University of Science and Technology of China(中国科学技术大学)
nuPlan-R: A Closed-Loop Planning Benchmark for Autonomous Driving via Reactive Multi-Agent Simulation
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Intelligent Transportation Thrust(智能交通方向);Robotics and Autonomous Systems Thrust(机器人与自主系统方向);Division of Emerging Interdisciplinary Areas(新兴跨学科领域部门)
TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology Beijing(北京科技大学);Baidu Inc.(百度公司)
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心);Robert Bosch GmbH(罗伯特·博世有限公司);OpenDriveLab at Shanghai AI Lab(上海人工智能实验室OpenDriveLab);NVIDIA Research(英伟达研究院);Nanyang Technological University(南洋理工大学);University of Toronto(多伦多大学);Vector Institute(向量研究院);Stanford University(斯坦福大学)
Dream to Drive with Predictive Individual World Model
School of Automation and Electrical Engineering, University of Science and Technology Beijing(北京科技大学自动化与电气工程学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Global Center for Automotive Performance Simulation(全球汽车性能模拟中心);Honda Research Institute, USA Inc.(美国本田研究院);Virginia Tech Transportation Institute(弗吉尼亚理工交通研究所)
See Tomorrow, Act Today: Foresight-Driven Autonomous Driving
School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Innovation Institute(上海创新研究院);Imperial College London(伦敦帝国理工学院);University of Surrey(萨里大学)
RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving
School of Information Technology, Monash University, Malaysia(墨尔本大学信息科技学院);Faculty of Information Technology, Monash University, Australia(墨尔本大学信息技术学院);TCS Research, India(印度TCS研究)
Agile Reactive Navigation for A Non-Holonomic Mobile Robot Using A Pixel Processor Array
Bristol Robotics Laboratory, The University of Bristol(布里斯托尔机器人实验室,布里斯托尔大学);Visual Information Laboratory, The University of Bristol(视觉信息实验室,布里斯托尔大学);Department of Electrical and Electronic Engineering, The University of Manchester(曼彻斯特大学电气与电子工程系);The Department of Computer Science and Engineering & The Department of Ophthalmology, The University of California, San Diego(加利福尼亚大学圣地亚哥分校计算机科学与工程系及眼科系);Hangzhou ATG Intelligent Technology Co. Ltd.(杭州ATG智能科技有限公司);Amazon(亚马逊公司);The University of California, San Diego(加利福尼亚大学圣地亚哥分校);The University of Manchester(曼彻斯特大学);The University of Bristol(布里斯托尔大学);Amazon, Seattle, USA(亚马逊公司(西雅图));The Department of Ophthalmology, The University of California, San Diego(加利福尼亚大学圣地亚哥分校眼科系)
Autonomous bot with ML-based reactive navigation for indoor environment
Vellore Institute of Technology(韦洛尔理工学院)
本地或反应导航是必不可少的自主移动机器人在室内环境中运行。诸如SLAM、计算机视觉的技术需要显著的计算能力,这增加了成本。同样,使用基本的方法使机器人容易受到不一致的行为。本文的目的是开发一种机器人,平衡成本和精度,通过使用机器学习预测最佳避障移动的基础上,从四个超声波传感器的距离输入,战略上安装在前面,左前方,右前方,和后面的机器人。底层硬件由Arduino Uno和Raspberry Pi 3B组成。机器学习模型首先在机器人收集的数据上进行训练。然后,Arduino不断轮询传感器并计算距离值,并且在紧急需要避免的情况下,Arduino会进行适当的机动。在其他情况下,传感器数据使用USB连接发送到Raspberry Pi,机器学习模型生成导航的最佳移动,然后发送到Arduino以相应地驱动电机。该系统安装在2-WD机器人底盘上,并在杂乱的室内环境中进行测试,结果令人印象深刻。
INTENT: An LSTM Framework for Vehicle Intention Prediction in Intersection Scenarios with Comprehensive Ablation Analysis
C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems(C-DRiVeS实验室:车载系统认知驾驶研究);Computer Science and Engineering Department - Faculty of Media Engineering and Technology - German University in Cairo(计算机科学与工程系-媒体工程与技术学院-埃及德国大学)
Learning collision risk proactively from naturalistic driving data at scale
Department of Transport & Planning(交通与规划系);Department of Engineering Systems and Services(工程系统与服务系);CityAI lab, Delft University of Technology(代尔夫特理工大学CityAI实验室)
PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning
School of Automation and Electrical Engineering, University of Science and Technology Beijing(北京科技大学自动化与电气工程学院);Xiaomi EV(小牛电动车);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室)
Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Training
Institute of Trustworthy Embodied AI, Fudan University(可信具身人工智能研究院,复旦大学);Shanghai Collaborative Innovation Center of Intelligent Visual Computing(上海智能视觉计算协同创新中心);The Hong Kong Polytechnic University(香港理工大学);NVIDIA
ADDT -- A Digital Twin Framework for Proactive Safety Validation in Autonomous Driving Systems
Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院);University of California, Irvine(加州大学欧文分校);Georgia Institute of Technology(佐治亚理工学院);South China University of Technology(华南理工大学)
An Autonomous Driving Model Integrated with BEV-V2X Perception, Fusion Prediction of Motion and Occupancy, and Driving Planning, in Complex Traffic Intersections