arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.03535cs.AIcs.RO
收藏

SenseRAG:利用主动查询构建面向基于 LLM 的自动驾驶的环境知识库

SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving

School of Information Technology, Monash University(莫纳什大学信息技术学院);University of Utah(犹他大学);Texas A&M University(德克萨斯农工大学)

Xuewen Luo, Fan Ding, Fengze Yang, Yang Zhou, Junnyong Loo, Hwa Hui Tew, Chenxi Liu

AI总结:

提出 SenseRAG 框架,通过主动式 RAG、多模态环境知识库与思维链提示增强 LLM 对自动驾驶场景的实时理解,并在真实 V2X 数据集上验证其感知与预测性能提升。

摘要:

本研究通过利用大型语言模型(LLM)的情境推理能力,解决自动驾驶(AD)中增强态势感知的关键需求。不同于依赖僵化、基于标签标注的传统感知系统,该研究将实时多模态传感器数据整合到统一的、LLM 可读的知识库中,使 LLM 能够动态理解并响应复杂驾驶环境。为克服 LLM 固有的延迟和模态限制,研究为自动驾驶设计了主动式检索增强生成(RAG),并结合思维链提示机制,以确保快速且上下文丰富的理解。使用真实车联万物(V2X)数据集开展的实验结果表明,感知和预测性能均获得显著提升,凸显了该框架在增强下一代 AD 系统安全性、适应性和决策能力方面的潜力。

英文摘要:

This study addresses the critical need for enhanced situational awareness in autonomous driving (AD) by leveraging the contextual reasoning capabilities of large language models (LLMs). Unlike traditional perception systems that rely on rigid, label-based annotations, it integrates real-time, multimodal sensor data into a unified, LLMs-readable knowledge base, enabling LLMs to dynamically understand and respond to complex driving environments. To overcome the inherent latency and modality limitations of LLMs, a proactive Retrieval-Augmented Generation (RAG) is designed for AD, combined with a chain-of-thought prompting mechanism, ensuring rapid and context-rich understanding. Experimental results using real-world Vehicle-to-everything (V2X) datasets demonstrate significant improvements in perception and prediction performance, highlighting the potential of this framework to enhance safety, adaptability, and decision-making in next-generation AD systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑