arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.16007cs.CVcs.AI
收藏

PhysicsMind: 为基础多模态大语言模型和世界模型中的物理推理和预测进行仿真与现实力学基准测试

PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models

Peking University(北京大学);Mohamed bin Zayed University of Artificial Intelligence(莫扎伊德大学人工智能学院);National University of Singapore(新加坡国立大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Science and Technology of China(中国科学技术大学);Cornell University(康奈尔大学);Hong Kong Polytechnic University(香港理工大学);City University of Hong Kong(香港城市大学)
Chak-Wing Mak, Guanyu Zhu, Boyi Zhang, Hongji Li, Xiaowei Chi, Kevin Zhang, Yichen Wu, Yangfan He, Chun-Kai Fan, Wentao Lu, Kuangzhi Ge, Xinyu Fang, Hongyang He… 展开作者

Chak-Wing Mak, Guanyu Zhu, Boyi Zhang, Hongji Li, Xiaowei Chi, Kevin Zhang, Yichen Wu, Yangfan He, Chun-Kai Fan, Wentao Lu, Kuangzhi Ge, Xinyu Fang, Hongyang He, Kuan Lu, Tianxiang Xu, Li Zhang, Yongxin Ni, Youhua Li, Shanghang Zhang

AI总结:

PhysicsMind是一个结合现实和仿真环境的统一基准,用于评估基础多模态大语言模型和世界模型在物理推理和预测中的能力。

摘要:

现代基础多模态大语言模型(MLLMs)和视频世界模型在数学、常识和视觉推理方面取得了显著进步,但其对底层物理的理解仍处于探索阶段。现有的基准测试试图衡量这一方面通常依赖于合成的视觉问题回答模板或专注于感知视频质量,这与测量视频是否符合物理定律无关。为了解决这种碎片化问题,我们引入了PhysicsMind,一个结合现实和仿真环境的统一基准,评估遵循三个经典原则的推理和生成:质心、杠杆平衡和牛顿第一定律。PhysicsMind包含两个主要任务:i)VQA任务,测试模型是否能从图像或短视频中推理并确定物理量和值;ii)视频生成(VG)任务,评估预测的运动轨迹是否遵循与地面真实值相同的质心、力矩和惯性约束。广泛范围的近期模型和视频生成模型在PhysicsMind上进行了评估,发现它们依赖于外观启发式方法,但往往违反基本力学。这些差距表明,当前的扩展和训练仍不足以实现稳健的物理理解,凸显了PhysicsMind作为物理感知多模态模型的专门测试平台。我们的数据将在接受后发布。

英文摘要:

Modern foundational Multimodal Large Language Models (MLLMs) and video world models have advanced significantly in mathematical, common-sense, and visual reasoning, but their grasp of the underlying physics remains underexplored. Existing benchmarks attempting to measure this matter rely on synthetic, Visual Question Answer templates or focus on perceptual video quality that is tangential to measuring how well the video abides by physical laws. To address this fragmentation, we introduce PhysicsMind, a unified benchmark with both real and simulation environments that evaluates law-consistent reasoning and generation over three canonical principles: Center of Mass, Lever Equilibrium, and Newton's First Law. PhysicsMind comprises two main tasks: i) VQA tasks, testing whether models can reason and determine physical quantities and values from images or short videos, and ii) Video Generation(VG) tasks, evaluating if predicted motion trajectories obey the same center-of-mass, torque, and inertial constraints as the ground truth. A broad range of recent models and video generation models is evaluated on PhysicsMind and found to rely on appearance heuristics while often violating basic mechanics. These gaps indicate that current scaling and training are still insufficient for robust physical understanding, underscoring PhysicsMind as a focused testbed for physics-aware multimodal models. Our data will be released upon acceptance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑