TALSC：面向基础设施辅助自动驾驶的时效性感知大小视觉语言模型协作
TALSC: Timeliness-Aware Large-Small VLM Collaboration for Infrastructure-Assisted Autonomous Driving
浏览论文内容
中文总结 AI 辅助
针对基础设施辅助自动驾驶中VLM协作的时效性问题，提出TALSC框架，通过李雅普诺夫漂移加估计惩罚算法优化调度，在nuScenes仿真中较最优基线实现Micro-F1最高12.6%的归一化提升。
中文摘要 AI 辅助
视觉语言模型（VLM）在自动驾驶（AD）系统中的部署受限于车载算力，使得车辆只能使用感知与推理能力有限的小型VLM（SVLM）。基础设施辅助自动驾驶通过与边缘服务器的大型VLM（LVLM）协作缓解了这一资源约束，但在动态车载环境中，下游任务的感官数据效用会快速衰减，信息时效性成为关键问题。为平衡LVLM带来的精度提升与延迟导致的时效性下降，我们提出了时效性感知大小VLM协作（TALSC）框架。具体而言，我们首先为VLM推理建模信息年龄（AoI）的演化，并刻画AoI、token长度与任务性能之间的耦合关系，以构建通用时效性度量。在此基础上，我们提出了TALSC在线调度算法；由于调度决策对未来时效性度量的影响存在延迟，且调度时的输出token数量未知，我们设计了李雅普诺夫漂移加估计惩罚算法，并提供了性能保证。在仿真中，我们首先基于nuScenes数据集进行案例研究以推导拟合的时效性度量，进一步表明TALSC在各类通信与计算设置下均优于基线，与性能最优的基线相比，在Micro-F1分数上实现了最高12.6%的归一化提升。
英文摘要
The deployment of Vision-Language Models (VLMs) in autonomous driving (AD) systems is constrained by on-board computing power, restricting vehicles to small VLMs (SVLMs) with limited perception and reasoning capabilities. Infrastructure-assisted AD alleviates this resource constraint by enabling collaboration with large VLMs (LVLMs) at edge servers. However, in dynamic vehicular environments, the utility of sensory data for downstream tasks decays rapidly, making timeliness of information a critical concern. To balance the accuracy gains of LVLMs with their latency-induced timeliness degradation, we develop a Timeliness-Aware Large-Small VLM Collaboration (TALSC) framework. Specifically, we first model the Age of Information (AoI) evolution for VLM inference and characterize the coupling among AoI, token length, and task performance to formulate a general timeliness metric. Building on this, we propose the TALSC online scheduling algorithm. Since scheduling decisions have a delayed impact on future timeliness metric and the output token number is unknown at scheduling time, we design a Lyapunov drift-plus-estimated-penalty algorithm and provides a guaranteed performance. In simulation, we first conduct a case study to derive a fitted timeliness metric based on nuScenes dataset, and further show that TALSC outperforms baselines under various communication and computing settings, achieving up to a 12.6\% normalized improvement in Micro-F1 score compared with the best-performing baseline.
发表机构
- School of Electronic and Information Engineering, Beijing Jiaotong University（北京交通大学电子与信息工程学院）
机构由 AI 辅助整理，请以论文原文为准。
