WorldAct:将单体3D世界激活为可交互的以对象为中心的场景
WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes
AI总结:
WorldAct通过多模态代理将静态生成的3D世界分解为可编辑的交互场景,支持对象级编辑和任务执行,保留全局一致性。
摘要:
最近基于生成场景合成的3D世界建模系统,如Marble,能够创建连贯且可探索的3D环境,但其输出通常是静态的单体资产,具有有限的可编辑性和物理交互能力。这限制了其在沉浸式内容创作和具身模拟中的应用,其中生成的世界必须被主动修改和操控。为解决这一挑战,我们提出了WorldAct,一个将静态生成的3D世界转换为可编辑和交互准备的场景的框架。WorldAct使用多模态代理指导场景分解,识别可操作对象,重建几何对齐的对象级网格以供交互,并通过3D修复恢复残留背景。所得到的场景支持对象级编辑、碰撞感知的操作和具身任务执行,同时保持全局场景一致性。实验表明,WorldAct比原始生成场景支持更丰富的交互场景,表明了通往可编辑和交互3D世界模型的实用路径。
英文摘要:
Recent 3D world modeling systems based on generative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation and embodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting. The resulting scenes support object-level editing, collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。