ReSTI：对STI-Bench的源基审计与修复

中文摘要 AI 辅助

时空基准测试仅在问题、源标注和答案选项指向同一物理量时才有效。我们对照官方ScanNet、Waymo和Omni6DPose源对STI-Bench进行了审计，发现了系统性的坐标系和时间戳错误、目标与时间指定不明确，以及键控选项与答案细节之间的不一致。我们提出了ReSTI，一种基于源的修订方法，在明确的目标、时间、坐标系、物理量和单位下重建每个可恢复的答案。源重建揭示了任务层面的几何失败：ScanNet Grounding忽略了标注与原始相机坐标系之间所需的对齐，而Orientation则在错误的平面上测量相机旋转。ReSTI用明确的、与源一致的几何定义替换了这些标签，并纠正了其他可由源验证的缺陷，包括在错误时间戳下评估的Waymo位姿。在2,064个遗留问题中，ReSTI保留了1,782个问题，并记录了282个有证据支持的排除项。因此，ReSTI为评估精确的视频时空推理提供了保守且可溯源的基础。项目页面：此https URL。

英文摘要

Spatial--temporal benchmarks are valid only when their questions, source annotations, and answer options identify the same physical quantity. We audit STI-Bench against the official ScanNet, Waymo, and Omni6DPose sources and find systematic coordinate-system and timestamp errors, under-specified targets and times, and disagreements between keyed options and answer details. We introduce ReSTI, a source-backed revision that reconstructs every recoverable answer under an explicit target, time, coordinate system, physical quantity, and unit. Source reconstruction reveals task-level geometric failures: ScanNet Grounding omits the required alignment between annotation and raw camera coordinate systems, while Orientation measures camera rotation on the wrong plane. ReSTI replaces these labels with explicit, source-consistent geometric definitions and corrects other source-verifiable defects, including Waymo poses evaluated at the wrong timestamp. Across 2,064 legacy questions, ReSTI retains 1,782 questions and records 282 evidence-backed exclusions. ReSTI therefore provides a conservative and source-traceable basis for evaluating precise video spatial--temporal reasoning. Project page: https://github.com/pengzhansun/ReSTI.

发表机构

• National University of Singapore（新加坡国立大学）
• University of Science and Technology of China（中国科学技术大学）

机构由 AI 辅助整理，请以论文原文为准。