arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01766cs.AI
收藏

SimWorlds: 一个用于动态3D场景创建的多智能体系统

SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation

Carnegie Mellon University(卡内基梅隆大学);Harvard University(哈佛大学);University of California, Merced(加州大学默塞德分校)

Chunjiang Liu, Xiaoyuan Wang, Haoyu Chen, Yizhou Zhao, Ming-Hsuan Yang, László A. Jeni

AI总结:

提出多智能体框架SimWorlds,通过规划-编码-审查流程和运行时状态检查工具,从文本生成动态4D场景,并引入基准4DBuildBench评估视觉保真度和物理一致性。

摘要:

LLM智能体越来越多地被用于以程序化方式将自然语言翻译成3D场景,但现有系统侧重于静态输出。仅从文本生成动态4D场景(其中液体流动、粒子发射、刚体级联和铰接机构运动)在很大程度上仍未探索,尽管它们作为可编辑内容以及视频生成和具身AI的物理基础训练数据具有价值。两个挑战使动态情况与静态文本到场景的工作区分开来:智能体必须在一个连贯的场景中共同协调空间布局、多个物理求解器、时间序列、相机和光照,并且从渲染视频中验证运动正确性比判断单张图像根本更难。我们提出SimWorlds:一个多智能体框架,从文本生成动态、可编辑的4D场景,具有Blender特定的程序化知识、驱动固定有序构建阶段序列的规划-编码-审查工作流、由确定性验证器强制执行的分层场景协议,以及捕获渲染图像无法揭示的机制故障的运行时状态检查工具套件。我们还引入了4DBuildBench,一个用于评估从文本提示生成的程序化动态3D场景的视觉保真度和物理一致性的基准。实验表明,SimWorlds优于先前的动态Blender生成基线。

英文摘要:

LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI. Two challenges set the dynamic case apart from static text-to-scene work: an agent must jointly coordinate spatial layout, multiple physics solvers, temporal sequencing, camera, and lighting in a single coherent scene, and verifying motion correctness from rendered video is fundamentally harder than judging a single image. We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text, with Blender-specific procedural knowledge, a planner-coder-reviewer workflow driving a fixed ordered sequence of construction stages, a layered scene protocol enforced by a deterministic verifier, and a runtime-state inspection tool suite that catches mechanism failures the rendered image cannot reveal. We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes generated from text prompts. Experiments show that SimWorlds outperforms prior dynamic Blender generation baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑