DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving
School of Information Science and Engineering, Lanzhou University(兰州大学信息科学与工程学院);The School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动性学院);The Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);DiDi, Beijing, China(滴滴出行);State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University(清华大学智能绿色车辆与移动性国家重点实验室)
Zhiye Wang, Yanbo Jiang, Rui Zhou, Bo Zhang, Fang Zhang, Zhenhua Xu, Yaqin Zhang, Jianqiang Wang
Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.
Sensor Fusion by Spatial Encoding for Autonomous Driving
Cho Chun Shik Graduate School of Mobility(赵春植 mobility 研究生院(注:该学院为KAIST下属学院,常见译法为赵春植移动研究生院));Division of Future Vehicle(未来车辆学部);Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院(KAIST))
ARETE: Attention-based Rasterized Encoding for Topology Estimation using HSV-transformed Crowdsourced Vehicle Fleet Data
Mercedes-Benz AG, Research \& Development, Sindelfingen, Germany Institute for Intelligent Systems, Esslingen University of Applied Sciences, Esslingen, Germany
AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving
Shanghai Jiao Tong University(上海交通大学);Cowa Robot(科沃机器人);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);The Chinese University of Hong Kong Shenzhen(香港中文大学(深圳))
TENET: Transformer Encoding Network for Effective Temporal Flow on Motion Prediction
Megvii Inc.(旷视科技有限公司);The Hong Kong University of Science and Technology(香港科技大学);Beihang University(北京航空航天大学);University of Michigan, Ann Arbor(密歇根大学安娜堡分校);Harbin Institute of Technology(哈尔滨工业大学);Tsinghua University(清华大学)
MTR-VP: Towards End-to-End Trajectory Planning through Context-Driven Image Encoding and Multiple Trajectory Prediction
Machine Intelligence, Interaction, and Imagination (Mi 3 ) Laboratory(机器智能、交互与想象实验室);University of California, Merced(加州大学默塞德分校);Laboratory for Intelligent & Safe Automobiles (LISA)(智能与安全汽车实验室);University of California, San Diego(加州大学圣地亚哥分校)
HENet: Hybrid Encoding for End-to-end Multi-task 3D Perception from Multi-view Cameras
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Chongqing Changan Automobile Co., Ltd.(重庆长安汽车股份有限公司);University of California, Merced(加州大学默塞德分校)
All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving
Simon Fraser University(西蒙·弗雷泽大学);Massachusetts Institute of Technology(麻省理工学院);University of Utah(犹他大学);University of Maryland, College Park(马里兰大学学院公园分校)
Towards High Efficient Long-horizon Planning with Expert-guided Motion-encoding Tree Search
The Chinese University of Hong Kong(香港中文大学);Macao Polytechnic University(澳门理工大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Southern University of Science and Technology(南方科技大学)
STAGE: STyle-controllable Action GEneration for personalized autonomous driving
Northwestern Polytechnical University(西北工业大学);School of Astronautics(航天学院);Research Center for Intelligent Robotics(智能机器人研究中心);National Key Laboratory of Aerospace Flight Dynamics(航空航天飞行动力学重点实验室)
JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving
Waymo(Waymo公司)
我们提出了JFP,一个联合未来预测模型,可以学习生成准确和一致的多智能体未来轨迹。对于这项任务,已经提出了许多不同的方法来捕获模型编码部分的社会交互,然而,在解码器和输出阶段中表示交互的关注要少得多。结果,预测的轨迹不一定彼此一致,并且经常导致不切实际的轨迹重叠。相比之下,我们提出了一个端到端的可训练模型,它直接学习结构化图形模型中的代理对之间的交互,以生成一致的未来轨迹。它在Waymo Open Motion Dataset(WOMD)上为交互式设置设置了新的最先进的结果。我们还研究了一个更复杂的多智能体设置为WOMD和一个更大的内部数据集,我们的方法显着提高了轨迹重叠指标,同时获得同等或更好的性能单智能体轨迹指标。
LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers
Institute for Intelligent Systems, Esslingen University of Applied Sciences(智能系统研究所,埃斯林根应用科学大学);Department of Computer Science, University of Freiburg(计算机科学系,弗赖堡大学)
LANet: A Lane Boundaries-Aware Approach For Robust Trajectory Prediction
Artificial Intelligence Creative Research Lab, ETRI University of Science(ETRI人工智能创意研究实验室,科学技术大学);Artificial Intelligence Creative Research Lab, ETRI Daejeon, South Korea(ETRI人工智能创意研究实验室,韩国大田)
Data-driven Camera and Lidar Simulation Models for Autonomous Driving: A Review from Generative Models to Volume Renderers
WMG, University of Warwick(华威大学WMG学院);School of Electronics, Electrical Engineering and Computer Science at Queen’s University of Belfast(贝尔法斯特女王大学电子、电气工程与计算机科学学院)
SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
Mercedes-Benz AG(梅赛德斯-奔驰集团);University of Tübingen(图宾根大学);Tübingen AI Center(图宾根人工智能中心);TU Munich(慕尼黑工业大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);University of Stuttgart(斯图加特大学);UCLA(加州大学洛杉矶分校)
GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Driving
School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence(北京交通大数据挖掘与具身智能重点实验室);Qcraft;Yanshan University(燕山大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);The University of Queensland(昆士兰大学)