探测驾驶视觉语言模型的可靠性：从不一致响应到基于现实的时序推理
Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning
AI总结：
本文研究了驾驶视觉语言模型的可靠性问题，通过引入FutureVQA数据集和自监督微调方法，提升模型在时序推理和一致性上的表现。
摘要：
一个可靠的驾驶助手应能基于从观察信息中推导出的时序相关推理提供一致的响应。在本工作中，我们探讨了当将视觉语言模型（VLMs）用作驾驶助手时，是否能够产生一致的响应并理解当前观察如何塑造未来结果，或者其输出仅仅是反映训练期间记忆的模式，而没有基于时序推理。尽管最近的努力已将VLMs整合到自动驾驶中，但先前的研究通常强调场景理解和指令生成，隐含地假设强大的视觉解释自然能够产生一致的未来推理，从而确保可靠的决策，这一主张我们进行了批判性审视。我们关注两个主要限制VLM在该设置下的可靠性的挑战：响应不一致性，其中微小的输入扰动会导致不同的答案，或者在某些情况下，响应退化为接近随机猜测；以及有限的时序推理，其中模型无法从当前观察中推理并对连续事件进行对齐，往往导致错误甚至矛盾的响应。此外，我们发现具有强大视觉理解能力的模型并不一定在需要时序推理的任务中表现最佳，这表明一种倾向于过度依赖预训练模式而非建模时序动态的趋势。为了解决这些问题，我们采用现有的评估方法，并引入了FutureVQA，一个由人工标注的基准数据集，专门用于评估未来场景推理。此外，我们提出了一种简单但有效的自监督微调方法，结合了链式推理，能够在不需时序标签的情况下提高一致性和时序推理能力。
英文摘要：
A reliable driving assistant should provide consistent responses based on temporally grounded reasoning derived from observed information. In this work, we investigate whether Vision-Language Models (VLMs), when applied as driving assistants, can response consistantly and understand how present observations shape future outcomes, or whether their outputs merely reflect patterns memorized during training without temporally grounded reasoning. While recent efforts have integrated VLMs into autonomous driving, prior studies typically emphasize scene understanding and instruction generation, implicitly assuming that strong visual interpretation naturally enables consistant future reasoning and thus ensures reliable decision-making, a claim we critically examine. We focus on two major challenges limiting VLM reliability in this setting: response inconsistency, where minor input perturbations yield different answers or, in some cases, responses degenerate toward near-random guessing, and limited temporal reasoning, in which models fail to reason and align sequential events from current observations, often resulting in incorrect or even contradictory responses. Moreover, we find that models with strong visual understanding do not necessarily perform best on tasks requiring temporal reasoning, indicating a tendency to over-rely on pretrained patterns rather than modeling temporal dynamics. To address these issues, we adopt existing evaluation methods and introduce FutureVQA, a human-annotated benchmark dataset specifically designed to assess future scene reasoning. In addition, we propose a simple yet effective self-supervised tuning approach with chain-of-thought reasoning that improves both consistency and temporal reasoning without requiring temporal labels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
