NICE FACT: 诊断和校准VLMs在运动物理定量推理中的能力
NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
AI总结：
本文研究VLMs在运动物理定量推理中的表现，提出NICE和FACT双诊断框架，分析视觉真实性、物理定律理解和时间定位，揭示模型在识别视觉前提和应用物理定律方面的不足。
摘要：
推导精确空间和物理洞察力是视觉语言模型（VLMs）的核心能力，但其在相关空间智能任务如物理推理中的表现较差仍是一个根本障碍。本文旨在深入理解VLMs如何感知物理世界并利用物理定律，同时评估模型置信度的可靠性。我们提出NICE和FACT双诊断范式，明确分解运动物理的定量推理：FACT诊断视觉真实性、物理定律理解和时间定位。NICE研究我们的新型邻域感知校准方法和新指标以评估和校准置信度可靠性。在6种最新最先进的VLMs上评估后，发现模型无法识别视觉前提或利用必要的物理定律以得出答案。本文强调并建立了标准化诊断范式，以指导开发忠实且物理基础的VLMs。
英文摘要：
The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks such as physical reasoning remain a fundamental barrier. The community critically lacks a scientific analysis revealing whether VLMs faithfully reach answers or plausibly make guesses. This work aims to provide a fundamental understanding of how VLMs perceive the physical world, and utilize physical laws, while assessing the reliability of model confidence. We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics: FACT diagnoses visual fidelity, physical law comprehension, and temporal grounding. NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability. Evaluated across 6 latest state-of-the-art VLMs, we uncover that models fail to identify visual preconditions or utilize necessary physical laws to reach answers. This work highlights and establishes a standardized diagnostic paradigm to guide the development of faithful, physically-grounded VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
