STRIDE-QA：用于城市驾驶场景时空推理的视觉问答数据集
STRIDE-QA: Visual Question Answering Dataset for Spatiotemporal Reasoning in Urban Driving Scenes
AI总结：
STRIDE-QA通过大规模视觉问答数据集提升自动驾驶中动态交通场景的时空推理能力，显著提升VLMs在空间定位和未来运动预测中的表现。
摘要：
视觉语言模型（VLMs）已被应用于自动驾驶，以支持复杂现实场景中的决策。然而，其在静态网络图像-文本对上的训练从根本上限制了理解和预测动态交通场景所需的精确时空推理能力。我们通过STRIDE-QA数据集填补这一关键空白，该数据集是基于第一人称视角的物理基础推理的大型视觉问答（VQA）数据集。该数据集由100小时的多传感器驾驶数据构建，捕捉了多样且具有挑战性的条件，STRIDE-QA是最大的用于城市驾驶时空推理的VQA数据集，提供了1600万对问答数据，覆盖27万个帧。通过密集的自动标注，包括3D边界框、分割掩码和多目标跟踪，该数据集通过三个新颖的问答任务支持基于对象和第一人称的推理，这些任务需要空间定位和时间预测。我们的基准测试表明，现有VLMs在预测一致性方面表现显著不足，得分接近零。相比之下，在STRIDE-QA上微调的VLMs在空间定位方面实现了55%的成功率，在未来运动预测的一致性方面达到了28%，相比通用VLMs的接近零得分。因此，STRIDE-QA为开发更可靠的VLMs奠定了全面的基础，以安全关键的自动驾驶系统。
英文摘要：
Vision-Language Models (VLMs) have been applied to autonomous driving to support decision-making in complex real-world scenarios. However, their training on static, web-sourced image-text pairs fundamentally limits the precise spatiotemporal reasoning required to understand and predict dynamic traffic scenes. We address this critical gap with STRIDE-QA, a large-scale visual question answering (VQA) dataset for physically grounded reasoning from an ego-centric perspective. Constructed from 100 hours of multi-sensor driving data in Tokyo, capturing diverse and challenging conditions, STRIDE-QA is the largest VQA dataset for spatiotemporal reasoning in urban driving, offering 16M QA pairs over 270K frames. Grounded by dense, automatically generated annotations including 3D bounding boxes, segmentation masks, and multi-object tracks, the dataset uniquely supports both object-centric and ego-centric reasoning through three novel QA tasks that require spatial localization and temporal prediction. Our benchmarks demonstrate that existing VLMs struggle significantly, with near-zero scores on prediction consistency. In contrast, VLMs fine-tuned on STRIDE-QA exhibit dramatic performance gains, achieving 55% success in spatial localization and 28% consistency in future motion prediction, compared to near-zero scores from general-purpose VLMs. Therefore, STRIDE-QA establishes a comprehensive foundation for developing more reliable VLMs for safety-critical autonomous systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
