arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00919cs.CVcs.RO
收藏

DriveCode: 针对基于LLM的自动驾驶的领域特定数值编码

DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

School of Information Science and Engineering, Lanzhou University(兰州大学信息科学与工程学院);The School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动性学院);The Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);DiDi, Beijing, China(滴滴出行);State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University(清华大学智能绿色车辆与移动性国家重点实验室)

Zhiye Wang, Yanbo Jiang, Rui Zhou, Bo Zhang, Fang Zhang, Zhenhua Xu, Yaqin Zhang, Jianqiang Wang

AI总结:

本文提出DriveCode,一种将数字表示为专用嵌入而非离散文本标记的新型数值编码方法,提升LLM在自动驾驶中的数值推理与解码效率。

摘要:

大型语言模型(LLMs)在自动驾驶中展现出巨大潜力。然而,将数字离散化为标记限制了精确的数值推理,无法反映数字在训练目标中的位置重要性,并使解码效率和数值精度难以兼顾。这些限制影响了传感器测量的处理和精确控制命令的生成,成为部署基于LLM的自动驾驶系统的基本障碍。本文引入DriveCode,一种新的数值编码方法,将数字表示为专用嵌入而非离散文本标记。DriveCode采用数字投影器将数字映射到语言模型的隐藏空间,使视觉和文本特征能够无缝整合到统一的多模态序列中。在OmniDrive、DriveGPT4和DriveGPT4-V2数据集上评估,DriveCode在轨迹预测和控制信号生成方面表现出色,证实了其在基于LLM的自动驾驶系统中的有效性。

英文摘要:

Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑