TinyDrive：面向自动驾驶的带选择性令牌路由的多尺度视觉问答
TinyDrive: Multiscale Visual Question Answering with Selective Token Routing for Autonomous Driving
AI总结：
针对自动驾驶VQA用VLM计算资源需求高、难部署的问题，提出轻量模型TinyDrive，通过多尺度编码器、令牌路由和序列优先级机制优化，在参数量更小的情况下于DriveLM基准取得SOTA语言理解性能。
摘要：
用于自动驾驶视觉问答（VQA）的视觉语言模型（VLM）通常需要大量计算资源，这给它们在资源受限的车辆中的部署带来了挑战。为解决这一问题，我们提出了TinyDrive，这是一款轻量且高效的VLM，适用于驾驶场景下的多视图VQA任务。我们的模型包含两个核心组件：多尺度视觉编码器，以及针对令牌和序列的双层优先级机制。多尺度编码器通过尺度注入和跨尺度门控，支持处理不同分辨率的多视图图像，以生成增强的视觉表征。在令牌层面，我们设计了令牌路由机制，可根据学习到的重要性分数动态选择并处理信息量最大的令牌。在序列层面，我们提出整合归一化损失、不确定性估计和多样性指标来构建序列分数，用于对序列优先级缓冲区中的样本进行排序和保留，分数更高的样本会被更频繁地选入训练。我们首先在自行构建的VQA数据集上评估TinyDrive，随后在公开基准DriveLM上进行测试，其取得了当前最优的语言理解性能。值得注意的是，尽管参数量大幅减少，它的BLEU-4和METEOR分数仍分别实现了11.1%和35.4%的相对提升。
英文摘要：
Vision Language Models (VLMs) employed for visual question-answering (VQA) in autonomous driving often require substantial computational resources that pose a challenge for their deployment in resource-constrained vehicles. To address this challenge, we introduce TinyDrive, a lightweight yet effective VLM for multi-view VQA in driving scenarios. Our model comprises two key components including a multiscale vision encoder and a dual-level prioritization mechanism for tokens and sequences. The multiscale encoder facilitates the processing of multi-view images at diverse resolutions through scale injection and cross-scale gating to generate enhanced visual representations. At the token level, we design a token routing mechanism that dynamically selects and process the most informative tokens based on learned importance scores. At the sequence level, we propose integrating normalized loss, uncertainty estimates, and a diversity metric to formulate sequence scores that rank and preserve samples within a sequence priority buffer. Samples with higher scores are more frequently selected for training. TinyDrive is first evaluated on our custom-curated VQA dataset, and it is subsequently tested on the public DriveLM benchmark, where it achieves state-of-the-art language understanding performance. Notably, it achieves relative improvements of 11.1% and 35.4% in BLEU-4 and METEOR scores, respectively, despite having a significantly smaller parameter count.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
