ExpressMind:一种用于高速公路运营的多模态预训练大语言模型
ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation
AI总结:
本文提出ExpressMind,一种针对高速公路运营的多模态预训练大语言模型,通过构建全栈数据集和双层预训练方法,提升事件检测、安全响应生成和复杂交通分析能力。
摘要:
当前高速公路运营依赖于基于规则的孤立模型,限制了跨系统知识的联合分析能力。同时,大语言模型(LLMs)在智能交通中的应用推动了交通模型从算法智能向认知智能发展。然而,通用LLMs无法有效理解高速公路领域中非常规场景的规则和因果关系。因此,本文构建了用于高速公路的预训练多模态大语言模型(MLLM)ExpressMind,作为智能高速公路运营的认知核心。本文构建了行业首个全栈高速公路数据集,涵盖交通知识文本、应急推理链和标注视频事件,以克服数据稀缺问题。本文提出了一种基于自监督训练和无监督学习的双层LLM预训练范式。此外,本文引入了图增强的RAG框架以动态索引高速公路知识库。为增强高速公路事故响应策略的推理能力,我们开发了与强化学习对齐的推理链(RL-CoT)机制,以强制模型推理与专家问题解决启发法之间的一致性。最后,ExpressMind集成了跨模态编码器以对视觉和文本通道下的动态特征序列进行对齐,使其能够理解视频和图像模态中的交通场景。在我们新发布的多模态高速公路基准上的广泛实验表明,ExpressMind在事件检测、安全响应生成和复杂交通分析方面全面优于现有基线。代码和数据可在:https://wanderhee.github.io/ExpressMind/上获得。
英文摘要:
The current expressway operation relies on rule-based and isolated models, which limits the ability to jointly analyze knowledge across different systems. Meanwhile, Large Language Models (LLMs) are increasingly applied in intelligent transportation, advancing traffic models from algorithmic to cognitive intelligence. However, general LLMs are unable to effectively understand the regulations and causal relationships of events in unconventional scenarios in the expressway field. Therefore, this paper constructs a pre-trained multimodal large language model (MLLM) for expressways, ExpressMind, which serves as the cognitive core for intelligent expressway operations. This paper constructs the industry's first full-stack expressway dataset, encompassing traffic knowledge texts, emergency reasoning chains, and annotated video events to overcome data scarcity. This paper proposes a dual-layer LLM pre-training paradigm based on self-supervised training and unsupervised learning. Additionally, this study introduces a Graph-Augmented RAG framework to dynamically index the expressway knowledge base. To enhance reasoning for expressway incident response strategies, we develop a RL-aligned Chain-of-Thought (RL-CoT) mechanism that enforces consistency between model reasoning and expert problem-solving heuristics for incident handling. Finally, ExpressMind integrates a cross-modal encoder to align the dynamic feature sequences under the visual and textual channels, enabling it to understand traffic scenes in both video and image modalities. Extensive experiments on our newly released multi-modal expressway benchmark demonstrate that ExpressMind comprehensively outperforms existing baselines in event detection, safety response generation, and complex traffic analysis. The code and data are available at: https://wanderhee.github.io/ExpressMind/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。