视觉语言模型无法推理物理变换
Vision Language Models Cannot Reason About Physical Transformation
AI总结：
本文通过构建ConservationBench基准，评估112个视觉语言模型在物理守恒任务上的表现，发现模型在动态场景中无法维持物理属性的变换不变性。
摘要：
理解物理变换是动态环境中推理的基础。虽然视觉语言模型（VLM）在具身应用中展现出潜力，但它们是否真正理解物理变换仍不清楚。我们引入了ConservationBench，用于评估守恒性——即物理量在变换下是否保持不变。该基准涵盖四种属性，包含成对的守恒/非守恒场景，我们生成并评估了112个VLM上的23,040个问题。结果揭示了系统性失败：性能接近随机水平，守恒任务上的改进伴随着控制任务上的下降。控制实验显示，模型存在强烈的文本先验偏向于不变性，但在守恒和非守恒场景中性能平衡时，模型对实际视觉内容的表现更差。时间分辨率、提示或精心采样的方法均无帮助。这些发现表明，当前VLM无法在动态场景中维持物理属性的变换不变性表示。
英文摘要：
Understanding physical transformations is fundamental for reasoning in dynamic environments. While Vision Language Models (VLMs) show promise in embodied applications, whether they genuinely understand physical transformations remains unclear. We introduce ConservationBench evaluating conservation -- whether physical quantities remain invariant under transformations. Spanning four properties with paired conserving/non-conserving scenarios, we generate and evaluate 23,040 questions across 112 VLMs. Results reveal systematic failure: performance remains near chance with improvements on conservation tasks accompanied by drops on controls. Control experiments show strong textual priors favoring invariance, yet models perform worse with actual visual content when performance is balanced across conserving and non-conserving scenarios. Neither temporal resolution, prompting, nor curated sampling helps. These findings show that current VLMs fail to maintain transformation-invariant representations of physical properties across dynamic scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
