arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03026cs.ROcs.AIcs.CLcs.CVcs.LG
收藏

LanguageMPC:大语言模型作为自动驾驶的决策者

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

Tsinghua University(清华大学);University of Hong Kong(香港大学)

Hao Sha, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, Mingyu Ding

AI总结:

针对现有自动驾驶系统在高级信息理解等方面的挑战,提出LanguageMPC方法,将大语言模型用作决策组件,设计认知路径与转换算法实现与低级控制器的集成,实验表明其在单车任务中优于基线且能处理多车协调等复杂场景。

摘要:

现有的基于学习的自动驾驶(AD)系统在理解高级信息、泛化到罕见事件以及提供可解释性方面面临挑战。为解决这些问题,本研究将大语言模型(LLMs)用作需要人类常识理解的复杂AD场景的决策组件。我们设计认知路径以实现LLMs的全面推理,并开发算法将LLM决策转化为可执行的驾驶指令。通过这种方法,LLM决策通过引导参数矩阵自适应与低级控制器无缝集成。大量实验表明,由于LLMs的常识推理能力,我们提出的方法不仅在单车任务中持续超越基线方法,还能处理复杂驾驶行为甚至多车协调。本文在安全、效率、泛化性和互操作性方面迈出了将LLMs用作复杂AD场景有效决策者的初步一步,希望能为该领域未来研究提供启发。项目页面:https://sites.google.com/view/llm-mpc

英文摘要:

Existing learning-based autonomous driving (AD) systems face challenges in comprehending high-level information, generalizing to rare events, and providing interpretability. To address these problems, this work employs Large Language Models (LLMs) as a decision-making component for complex AD scenarios that require human commonsense understanding. We devise cognitive pathways to enable comprehensive reasoning with LLMs, and develop algorithms for translating LLM decisions into actionable driving commands. Through this approach, LLM decisions are seamlessly integrated with low-level controllers by guided parameter matrix adaptation. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination, thanks to the commonsense reasoning capabilities of LLMs. This paper presents an initial step toward leveraging LLMs as effective decision-makers for intricate AD scenarios in terms of safety, efficiency, generalizability, and interoperability. We aspire for it to serve as inspiration for future research in this field. Project page: https://sites.google.com/view/llm-mpc

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑