ExeWorldBench:基于代码的可执行3D世界建模基准测试
ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
AI总结:
ExeWorldBench是首个全面评估世界理解与可执行实现的基准测试,含三个递进任务,实验揭示前沿模型在可执行世界建模上的显著局限,为相关系统开发提供洞见。
摘要:
编码智能体正逐渐从仅复现世界外观,向构建可执行世界(捕捉世界运行机制)发展。这一转变催生了可执行世界建模,它整合了两大核心能力:一是世界理解,即推理世界底层的结构、力学与因果原理;二是可执行实现,即将该理解转化为基于代码的系统,其行为由物理交互产生。然而,现有基准测试仅提供碎片化评估,聚焦于物理推理、视觉保真度或行为正确性,未能充分评估这些能力的整合情况。为填补这一空白,我们推出ExeWorldBench,据我们所知,这是首个通过基于代码的3D世界构建,全面评估世界理解与可执行实现的基准测试。ExeWorldBench包含三个难度递进的任务:机制可执行性任务,评估从视觉观测实现单个机制的能力;组合可执行性任务,扩展至具有空间、力学和因果依赖的多阶段系统;以及意图驱动的可执行世界合成任务,要求从高层目标和物理需求构建可执行世界。对前沿模型的大量实验揭示了可执行世界建模存在的显著局限,为开发更具物理基础和因果一致性的世界构建系统提供了洞见。
英文摘要:
Coding agents are increasingly evolving beyond reproducing what a world looks like toward constructing executable worlds that capture how a world works. This shift calls for executable world modeling, which integrates two core capabilities: world understanding, reasoning about the underlying structural, mechanical, and causal principles of a world, and executable realization, translating that understanding into a code-based system whose behavior emerges from physical interactions. However, existing benchmarks provide only fragmented evaluations centered on physical reasoning, visual fidelity, or behavioral correctness, leaving the integration of these capabilities insufficiently assessed. To bridge this gap, we introduce ExeWorldBench, to our knowledge the first benchmark to comprehensively evaluate world understanding and executable realization through code-based 3D world construction. ExeWorldBench comprises three progressively challenging tasks: Mechanism Executability, which evaluates the realization of individual mechanisms from visual observations; Compositional Executability, which extends to multi-stage systems with spatial, mechanical, and causal dependencies; and Intent-Driven Executable World Synthesis, which requires constructing executable worlds from high-level goals and physical requirements. Extensive experiments on frontier models reveal substantial limitations in executable world modeling, offering insights into the development of more physically grounded and causally coherent world construction systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。