arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.04515cs.CV
收藏

从先验到感知:将视频大语言模型锚定于物理现实

From Priors to Perception: Grounding Video-LLMs in Physical Reality

Shanghai Jiao Tong University(上海交通大学)

Zicheng Zhao, Chaofan Gan, Shijie Li, Weiyao Lin

AI总结:

针对视频大语言模型因语义先验主导而忽视视觉证据的物理判断失败,提出PriorPair配对基准与PhyAR轻量训练框架,通过视觉锚定推理链和配对监督,在不改架构下显著提升物理推理并降低先验偏差。

摘要:

视频大语言模型(Video-LLMs)在通用视频理解方面表现出色,但往往基于事件预期而非观察结果来做出物理判断。我们发现,它们不仅会将物理上不可能的事件合理化,而且在物理上合理却反直觉的场景中,尽管存在清晰的矛盾视觉证据,仍会错误地报告预期结果。我们首次将这些失败统一归因于语义先验主导(Semantic Prior Dominance, SPD):即语义预期压倒了冲突的视觉证据。为了诊断并克服这些失败,我们引入了PriorPair,一个高保真配对视频基准,涵盖八个物理类别中的冲突。其类别导向的构建方式将先验一致的事件与语义匹配但先验冲突的对应事件配对。我们进一步提出了物理锚定推理器(Physics-Anchored Reasoner, PhyAR),一个轻量级训练框架。其视觉锚定推理链将物理归因和判断明确锚定于视觉观察,而配对数据绑定则通过配对监督强化对决定性事件差异的学习。使用PhyAR进行标准LoRA微调,在无需修改架构的情况下,显著提升了不同视频大语言模型骨干网络的物理推理能力,并减少了先验驱动的判断偏差。这些提升可扩展到外部物理基准,同时基本保持通用视频理解能力。

英文摘要:

Video Large Language Models (Video-LLMs) excel in general video understanding but often base physical judgments on event expectations rather than observations. We find that they not only rationalize physically impossible events, but also incorrectly report expected outcomes in physically plausible yet counter-intuitive scenarios, despite clear contradictory visual evidence. We provide the first unified account of these failures as Semantic Prior Dominance (SPD): semantic expectations override conflicting visual evidence. To diagnose and overcome these failures, we introduce PriorPair, a high-fidelity paired video benchmark covering both conflicts across eight physical categories. Its category-directed construction pairs prior-aligned events with semantically matched but prior-conflicting counterparts. We further propose Physics-Anchored Reasoner (PhyAR), a lightweight training framework. Its Visually Anchored Reasoning Chain grounds physical attribution and judgment in explicit visual observations, while Paired Data Binding strengthens learning of decisive event differences through matched-pair supervision. Standard LoRA fine-tuning with PhyAR substantially improves physical reasoning and reduces prior-driven judgment bias across different Video-LLM backbones without architectural modifications. These gains extend to an external physical benchmark while largely preserving general video understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑