通过合成视频评估VLMs的情境与空间意识
Assessing Situational and Spatial Awareness of VLMs with Synthetically Generated Video
AI总结：
本文提出一个合成基准，评估VLMs在区分暴力与无害活动、跨视角角色绑定及细粒度轨迹对齐方面的性能，发现其表现仅略高于随机水平，且稳定颜色线索仅部分缓解了问题。
摘要：
视觉语言模型（VLMs）中的空间推理在语义依赖于微妙的时间或几何线索时仍显得脆弱。我们引入了一个合成基准，测试两种互补技能：情境意识（识别互动是否有害或无害）和空间意识（跟踪谁对谁做什么，以及推理相对位置和运动）。通过最小的视频对，我们测试了三个挑战：区分暴力与无害活动、在不同视角下绑定攻击者角色、以及判断细粒度轨迹对齐。尽管我们在无训练设置下评估了最近的VLMs，该基准适用于任何视频分类模型。结果显示，在所有任务中性能仅略高于随机水平。一个简单的辅助手段，稳定的颜色线索，部分减少了攻击者角色的混淆，但并未解决根本的弱点。通过发布数据和代码，我们旨在提供可重复的诊断，并激发对轻量级空间先验的探索，以补充大规模预训练。
英文摘要：
Spatial reasoning in vision language models (VLMs) remains fragile when semantics hinge on subtle temporal or geometric cues. We introduce a synthetic benchmark that probes two complementary skills: situational awareness (recognizing whether an interaction is harmful or benign) and spatial awareness (tracking who does what to whom, and reasoning about relative positions and motion). Through minimal video pairs, we test three challenges: distinguishing violence from benign activity, binding assailant roles across viewpoints, and judging fine-grained trajectory alignment. While we evaluate recent VLMs in a training-free setting, the benchmark is applicable to any video classification model. Results show performance only slightly above chance across tasks. A simple aid, stable color cues, partly reduces assailant role confusions but does not resolve the underlying weakness. By releasing data and code, we aim to provide reproducible diagnostics and seed exploration of lightweight spatial priors to complement large-scale pretraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
