arXiv 2605.201652026-05-19cs.CV

CaMo：基于摄像机运动的视觉-语言模型评估与训练

CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models

Department of Electrical and Computer Engineering, University of Washington, Seattle, USA（华盛顿大学电气与计算机工程系）

Hsiang-Wei Huang, Junbin Lu, Kuang-Ming Chen, Jianxu Shangguan, Cheng-Yen Yang, Jenq-Neng Hwang

备注：: Code and model available at https://github.com/hsiangwei0903/CaMo

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种基于摄像机运动的视觉-语言模型评估与训练方法CaMo，通过要求模型生成显式的空间叙述并进行推理，揭示了现有空间视觉-语言模型在空间认知方面的不足，并展示了CaMo在空间叙述评估和直接空间问题回答准确性上的一致表现。

摘要：

视觉-语言模型（VLMs）在空间问答基准测试中表现出色，但尚不清楚这种表现是否反映了真正的空间智能。我们证明现有空间VLMs缺乏基本的摄像机运动理解，这是空间认知的关键组成部分。我们提出了空间叙述评分（SNS），一种评估框架，要求VLMs生成显式的空间叙述，捕捉场景语义和摄像机运动，随后使用冻结的代理LLM进行推理。在SNS下，最先进的空间VLMs在直接问答准确性高时，却在评估中表现出显著的性能下降。为解决这一差距，我们引入了CaMo，一种基于摄像机运动的VLM，其在SNS评估和直接空间问答准确性上均表现出一致的性能。我们的结果强调了显式空间叙述外部化在评估具有可转移3D空间理解的VLMs中的重要性。我们的代码、数据和模型可在https://github.com/hsiangwei0903/CaMo上获得。

英文摘要：

Vision-Language Models (VLMs) achieve strong performance on spatial question answering benchmarks, yet it remains unclear whether such gains reflect genuine spatial intelligence. We show that existing spatial VLMs lack basic camera motion understanding, a key component of spatial cognition. We propose the Spatial Narrative Score (SNS), an evaluation framework that requires VLMs to generate explicit spatial narratives capturing both scene semantics and camera motion, followed by reasoning with a frozen proxy LLM. Under SNS, state-of-the-art spatial VLMs exhibit significant performance degradation despite high direct question answering accuracy. To address this gap, we introduce CaMo, a camera motion grounded VLM that achieves consistent performance across SNS evaluation and direct spatial question answering accuracy. Our results highlight the importance of explicit spatial narrative externalization for evaluating VLMs with transferable 3D spatial understanding. Our code, data, and model is available at https://github.com/hsiangwei0903/CaMo

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。