Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% of benchmark runs before fallback. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics. Our code is available https://github.com/TIGER-AI-Lab/VisPhyWorld
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Zhongguancun Academy(中关村学院);Tsinghua University(清华大学);Helsinki University(赫尔辛基大学)
Probing the effectiveness of World Models for Spatial Reasoning through Test-time Scaling
MILA – Quebec AI Institute(魁北克人工智能研究所);Polytechnique Montréal(蒙特利尔理工学院);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Institute for Machine Learning, Johannes Kepler University Linz(林茨约瑟夫·夫兰克大学机器学习研究所);Nankai University(南开大学)
Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning
Duncan of Jordanstone College of Art & Design (DJCAD), University of Dundee(邓迪大学邓迪艺术与设计学院(DJCAD));Guangzhou Institute of Science and Technology (GZIST)(广州科学技术研究院);Faculty of Arts, Xiamen University(厦门大学艺术学院);University of Birmingham(伯明翰大学)
Probing Mechanical Reasoning in Large Vision Language Models
Johns Hopkins University(约翰霍普金斯大学);University of North Carolina Chapel Hill(北卡罗来纳大学教堂山分校);University of Michigan(密歇根大学);University of California San Diego(加州大学圣地亚哥分校);Carnegie Mellon University(卡内基梅隆大学)
Absurd World: A Simple Yet Powerful Method to Absurdify the Real-world for Probing LLM Reasoning Capabilities
The Nueva School(新维学校);University of Southern California(南加州大学);Notre Dame College(诺特大学);Arizona State University(亚利桑那州立大学);National University of Singapore(新加坡国立大学)
From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
School of Information Science, Beijing Language and Culture University(北京语言大学信息科学学院);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
Answer Probing-Guided Search for Diverse Solution Exploration of LLMs
University of Science and Technology of China(中国科学技术大学);Zhongguancun Academy(中关村学院);Alibaba Group(阿里巴巴集团);Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学)
Coherence Under Commitment: Probing Generalization and Vacuous Memorization in LLM Logical Reasoning
Department of Computer Science, Informatics Institute, Istanbul Technical University(伊斯坦布尔技术大学信息学研究所计算机科学系);School of Information Technology, Deakin University(迪肯大学信息技术学院)
Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales
University of Central Florida(中央佛罗里达大学);University of California, Santa Cruz(加州大学圣克鲁兹分校);Columbia University(哥伦比亚大学);Amazon Research(亚马逊研究);University of California, Berkeley(加州大学伯克利分校)