arXiv 2603.211142026-08-17cs.CV

CVT-Bench：反事实视角变换揭示多模态大语言模型中不稳定的空间表征

CVT-Bench: Probing Spatial-State Integrity through Counterfactual Viewpoint Transformations

CSSE Department, Auburn University（计算机科学与工程系，阿伯杜大学）

Shanmukha Vellamcheti, Uday Kiran Kothapalli, Disharee Bhowmick, Sathyanarayanan N. Aakur

备注：: Reframed CVT-Bench around spatial-state integrity, added real-world benchmark derived from 3dSGG, context controls and other minor changes. 21 pages, 10 figures, 15 tables. Project page: https://shanmukha-here.github.io/CVT-Bench

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

CVT-Bench通过控制诊断基准评估多模态大语言模型在反事实视角变换下的关系一致性，发现尽管单视角准确率高，但系统在反事实视角变换下存在系统性退化，揭示了表征结构对反事实空间推理的重要性。

摘要：

多模态大语言模型（MLLMs）在单视角空间推理任务中表现出色，但其在反事实视角变换下的空间状态表示稳定性仍不明确。本文引入了一个受控诊断基准，评估在不重新渲染图像的情况下，假设相机轨道变换下的关系一致性。在100个合成场景和6000个关系查询中，我们测量了视角一致性、360度循环一致性以及连续变换中的关系稳定性。尽管单视角准确率高，最先进的MLLMs在反事实视角变换下表现出系统性退化，频繁违反循环一致性和关系稳定性快速衰减。我们进一步评估了多种输入表示、视觉输入、文本边界框和结构化场景图，并表明增加表征结构可以提高稳定性。我们的结果表明，单视角空间准确性高估了诱导空间表征的鲁棒性，且表征结构在反事实空间推理中起关键作用。

英文摘要：

Multimodal large language models (MLLMs) perform strongly on isolated spatial tasks, but whether their predictions remain persistent and mutually coherent across viewpoints and competing scenes is unclear. We formalize this behavioral property as spatial-state integrity and introduce CVT-Bench, a factorial diagnostic suite spanning two domains (CVT-Synthetic, CVT-Real), four context regimes (CVT-Isolated, irrelevant filler, attribute filler, CVT-Competing), three representations (Image, Text/BBox, Scene Graph), and ten viewpoint conditions (nine azimuths from $0^\circ$ to $360^\circ$ at $45^\circ$ increments, plus Top) with target views withheld. Across 200 scenes and 11,833 relational queries, we measure counterfactual accuracy, cycle consistency, the normalized survival coefficient, spatial realizability, and context-specific interference. Five state-of-the-art MLLMs exhibit rapid persistence loss and frequently produce jointly unrealizable states despite high local accuracy, with failures amplified by competing contexts and natural-scene complexity. Matched irrelevant and attribute filler controls show that prompt length, position, or loss of scene access alone are insufficient explanations. On average, Text/BBox improves accuracy, persistence, and realizability, while Scene Graph provides complementary gains; neither eliminates the instability. Thus, isolated spatial accuracy substantially overestimates robustness, establishing spatial-state integrity as a distinct evaluation target. The complete benchmark and codebase will be publicly released.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。