A knowledge-augmented dataset of high-risk driving scenarios with LLM annotations for autonomous driving
Korea Advanced Institute of Science and Technology(韩国科学技术院);Fudan University(复旦大学);Massachusetts Institute of Technology(麻省理工学院);Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学)
Heye Huang, Jingguang Li, Zhiyuan Zhou, Paul Liang, Mingyu Wu, Kitae Jang, Jianqiang Wang
Safe autonomous driving requires both rapid responses to common high-risk events and deeper reasoning over rare, extreme long-tail scenarios in traffic safety. These scenarios are severely under-represented in naturalistic driving data, and existing trajectory and language-augmented datasets seldom provide high-risk event labels, semantic annotations, and verifiable safety signals. Here we present K-Risk, a knowledge-augmented dataset that combines structured driving trajectories with large language model generated semantic annotations for safety-critical driving scenarios. K-Risk integrates 20 human-driven and autonomous-vehicle trajectory datasets from Europe, China, and the United States, covering highways, urban freeways, intersections, and roundabouts. Using a unified risk-centric extraction pipeline, K-Risk curates 31,398 high-risk events, together with a 1,036-event extreme subset of near-collision cases. Each event is released as a synchronized trajectory, metadata, and language triplet containing structured scenario descriptions, abnormal-behavior notifications, and, for a representative subset, causal risk analyses and action recommendations validated through a closed-loop simulator with iterative reflection. By combining multi-dimensional risk annotations, interpretable language supervision, and verifiable decisions, K-Risk bridges structured traffic trajectories, semantic reasoning, and decision supervision, providing a standardized foundation for developing and evaluating next-generation risk-aware autonomous driving agents.
doPlan: A Variable-Horizon Dataset for Multi-Stage Language-Conditioned Planning in Autonomous Driving
Laboratory for Intelligent & Safe Automobiles (LISA), University of California, San Diego(加州大学圣迭戈分校智能与安全汽车实验室(LISA));Machine Intelligence, Interaction, and Imagination (Mi) Laboratory, University of California, Merced(加州大学默塞德分校机器智能、交互与想象实验室(Mi))
Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning
University of California, Berkeley(加州大学伯克利分校);Zenseact(森赛克特);Chalmers University of Technology(查尔姆斯理工大学);Peking University(北京大学);Google Research(谷歌研究院)
DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy
School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动学院);Didi Global(滴滴出行);Department of Civil and Environmental Engineering, University of Michigan(密歇根大学土木与环境工程系);Lab for High Technology, Tsinghua University(清华大学高科技实验室);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院)
Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey
Alexander Thamm GmbH(亚历山大·塔姆有限公司);Capgemini Engineering(凯捷工程公司);Continental AG(大陆集团);Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)(德国人工智能研究中心);Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)(德国航空航天中心);e:fs TechHub GmbH(e:fs技术中心有限公司);fortiss GmbH(fortiss有限公司);Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. (FOKUS & IAIS)(弗劳恩霍夫应用研究促进协会);FZI Forschungszentrum Informatik(FZI信息技术研究中心);Robert Bosch GmbH(罗伯特·博世有限公司);Universität des Saarlandes(萨尔大学);Valeo Schalter und Sensoren GmbH(法雷奥开关与传感器有限公司)