In this paper, we explore the potential of using a large language model (LLM) to understand the driving environment in a human-like manner and analyze its ability to reason, interpret, and memorize when facing complex scenarios. We argue that traditional optimization-based and modular autonomous driving (AD) systems face inherent performance limitations when dealing with long-tail corner cases. To address this problem, we propose that an ideal AD system should drive like a human, accumulating experience through continuous driving and using common sense to solve problems. To achieve this goal, we identify three key abilities necessary for an AD system: reasoning, interpretation, and memorization. We demonstrate the feasibility of employing an LLM in driving scenarios by building a closed-loop system to showcase its comprehension and environment-interaction abilities. Our extensive experiments show that the LLM exhibits the impressive ability to reason and solve long-tailed cases, providing valuable insights for the development of human-like autonomous driving. The related code are available at https://github.com/PJLab-ADG/DriveLikeAHuman .
Rethinking Imitation-based Planner for Autonomous Driving
Hong Kong University of Science and Technology(香港科技大学);Lotus Technology Ltd.(路特斯科技有限公司);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detection
Hong Kong University of Science and Technology(香港科技大学);Tongji University(同济大学);Shanghai Research Institute for Intelligent Autonomous Systems(上海智能自主系统研究院)
Rethinking Range View Representation for LiDAR Segmentation
Shanghai AI Laboratory(上海人工智能实验室);National University of Singapore(新加坡国立大学);Hochschule Bremerhaven(不来梅哈芬应用技术大学);The University of Hong Kong(香港大学);ShanghaiTech University(上海科技大学);The Chinese University of Hong Kong(香港中文大学);Nanyang Technological University(南洋理工大学)
Enabling Level-4 Autonomous Driving on a Single $1k Off-the-Shelf Card
Perceptin LLC(Perceptin有限责任公司);Northeastern Univ(东北大学);COCOPLE LLC(COCOPLE有限责任公司);College of William and Mary(威廉玛丽学院);North Carolina State University(北卡罗来纳州立大学)
How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity
School of Computer Science and Technology Xi’an Jiaotong University China(西安交通大学计算机科学与技术学院);Shanghai AI Laboratory(上海人工智能实验室);MOE KLINNS Lab Xi’an Jiaotong University China(西安交通大学教育部KLINNS实验室)
DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy
College of Automotive Engineering, Jilin University(汽车工程学院,吉林大学);National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University(汽车底盘集成与生物力学国家重点实验室,吉林大学)