TS-VLM：面向多视角驾驶推理中视觉语言模型的文本引导SoftSort池化
TS-VLM: Text-Guided SoftSort Pooling for Vision-Language Models in Multi-View Driving Reasoning
AI总结：
针对现有视觉语言模型在自动驾驶多视角推理中计算开销大、多视角数据整合效率低的问题，提出集成文本引导SoftSort池化模块的轻量级模型TS-VLM，在DriveLM基准上性能优于现有最优模型且大幅降低计算成本。
摘要：
视觉语言模型（VLM）通过利用多模态融合来增强场景感知、推理与决策能力，在推动自动驾驶发展方面展现出显著潜力。尽管具备潜力，现有模型仍存在计算开销大、多视角传感器数据整合效率低的问题，导致其无法在安全关键型自动驾驶应用中实现实时部署。为解决这些缺陷，本文致力于设计一款名为TS-VLM的轻量级VLM，该模型集成了全新的文本引导SoftSort池化（TGSSP）模块。TGSSP借助输入查询的语义，对来自多个视角的视觉特征进行排序与融合，无需依赖高成本的注意力机制即可实现动态的、感知查询的多视角聚合。这种设计能够根据查询自适应地优先处理语义相关的视角，从而提升自动驾驶多视角推理中的上下文准确性。在DriveLM基准上的大量评估表明，一方面，TS-VLM的表现优于当前最优模型，其BLEU-4得分为56.82，METEOR得分为41.91，ROUGE-L得分为74.64，CIDEr得分为3.39；另一方面，TS-VLM最多可降低90%的计算成本，其中最小版本仅包含2010万个参数，更适合在自动驾驶车辆中进行实时部署。
英文摘要：
Vision-Language Models (VLMs) have shown remarkable potential in advancing autonomous driving by leveraging multi-modal fusion in order to enhance scene perception, reasoning, and decision-making. Despite their potential, existing models suffer from computational overhead and inefficient integration of multi-view sensor data that make them impractical for real-time deployment in safety-critical autonomous driving applications. To address these shortcomings, this paper is devoted to designing a lightweight VLM called TS-VLM, which incorporates a novel Text-Guided SoftSort Pooling (TGSSP) module. By resorting to semantics of the input queries, TGSSP ranks and fuses visual features from multiple views, enabling dynamic and query-aware multi-view aggregation without reliance on costly attention mechanisms. This design ensures the query-adaptive prioritization of semantically related views, which leads to improved contextual accuracy in multi-view reasoning for autonomous driving. Extensive evaluations on the DriveLM benchmark demonstrate that, on the one hand, TS-VLM outperforms state-of-the-art models with a BLEU-4 score of 56.82, METEOR of 41.91, ROUGE-L of 74.64, and CIDEr of 3.39. On the other hand, TS-VLM reduces computational cost by up to 90%, where the smallest version contains only 20.1 million parameters, making it more practical for real-time deployment in autonomous vehicles.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
