arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.08236cs.CVcs.AIcs.CL
收藏

何时想象以及想象多少:基于世界模型的自适应测试时缩放用于视觉空间推理

When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning

University of North Carolina, Chapel Hill(北卡罗来纳大学教堂山分校);Nanyang Technological University(南洋理工大学)

Shoubin Yu, Yue Zhang, Zun Wang, Jaehong Yoon, Huaxiu Yao, Mingyu Ding, Mohit Bansal

AI总结:

本文提出自适应测试时框架AVIC/AVIC-R,通过世界模型选择性调用和缩放视觉想象,在空间推理中平衡准确性与效率,超越GPT-4o等基线。

摘要:

尽管多模态大语言模型(MLLMs)取得了快速进展,但当正确答案取决于场景在未见或替代视角下的外观时,视觉空间推理仍然不可靠。最近的工作通过使用世界模型进行视觉想象来增强推理,但诸如想象何时真正必要、多少想象有益、以及何时想象有害等问题仍知之甚少。在实践中,无差别的想象可能会增加计算量,甚至通过引入误导性证据而降低性能。在这项工作中,我们深入分析了作为空间推理可控资源的测试时视觉想象。我们首先研究静态视觉证据何时足够,想象何时改进推理,以及过度或不必要的想象如何影响准确性和效率。为了支持这一分析,我们随后引入了AVIC,一个基于世界模型的自适应测试时框架,该框架在选择性调用和缩放视觉想象之前,明确推理当前视觉证据的充分性。最后,为了进一步学习这种门控和规划行为,而无需任何关于何时想象以及想象多少的标注,我们引入了AVIC-R,它通过来自QA正确性奖励和想象成本惩罚的GRPO来训练策略。在空间推理基准(SAT, MMSI)和具身导航基准(R2R)上,我们的结果揭示了想象至关重要、边际或有害的明确场景,并表明选择性控制可以匹配或超越固定想象策略,同时大幅减少世界模型调用和语言标记。我们的AVIC-R超越了包括GPT-4o和GPT-4.1在内的强大专有基线,同时调用世界模型的频率更低。总体而言,我们的发现强调了分析和控制测试时想象对于高效可靠的空间推理的重要性。

英文摘要:

Despite rapid progress in MLLMs, visual spatial reasoning remains unreliable when correct answers depend on how a scene would appear under unseen or alternative viewpoints. Recent work addresses this by augmenting reasoning with world models for visual imagination, but questions such as when imagination is actually necessary, how much of it is beneficial, and when it becomes harmful, remain poorly understood. In practice, indiscriminate imagination can increase computation and even degrade performance by introducing misleading evidence. In this work, we present an in-depth analysis of test-time visual imagination as a controllable resource for spatial reasoning. We first study when static visual evidence is sufficient, when imagination improves reasoning, and how excessive or unnecessary imagination affects accuracy and efficiency. To support this analysis, we then introduce AVIC, an adaptive test-time framework with world models that explicitly reasons about the sufficiency of current visual evidence before selectively invoking and scaling visual imagination. Finally, to further learn this gating and planning behavior without any annotation of when and how much to imagine, we introduce AVIC-R, which trains the policy via GRPO from QA-correctness rewards and penalties by imagination cost. Across spatial reasoning benchmarks (SAT, MMSI) and an embodied navigation benchmark (R2R), our results reveal clear scenarios where imagination is critical, marginal, or detrimental, and show that selective control can match or outperform fixed imagination strategies with substantially fewer world-model calls and language tokens. Our AVIC-R surpasses strong proprietary baselines including GPT-4o and GPT-4.1 while invoking the world model less often. Overall, our findings highlight the importance of analyzing and controlling test-time imagination for efficient and reliable spatial reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑