HOLODECK 2.0：基于视觉-语言的3D世界生成与编辑
HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editing
AI总结：
HOLODECK 2.0通过视觉-语言模型生成高质量3D场景并支持交互式编辑，提升游戏设计效率。
摘要：
3D场景生成在游戏、艺术创作、虚拟现实等多个领域中发挥着关键作用。然而，当前的3D场景设计仍然严重依赖创作者大量的手动工作，而现有的自动化方法在生成开放域场景或支持灵活编辑方面存在困难。为了解决这些挑战，我们引入了HOLODECK 2.0，一种先进的基于视觉-语言的3D世界生成框架，支持基于人类反馈的交互式场景编辑。HOLODECK 2.0可以生成多样且风格丰富的3D场景（例如写实、卡通、动画和赛博朋克风格），这些场景在细粒度输入描述上具有高度语义保真度，适用于室内外和开放域环境。HOLODECK 2.0利用视觉-语言模型（VLMs）来识别和解析场景所需对象，并通过最先进的3D生成模型生成相应的高质量资产。然后，HOLODECK 2.0通过从VLMs中推导出的空间约束进行迭代，以实现语义一致且物理上合理的布局。人类和模型评估均表明，HOLODECK 2.0能够有效生成高质量的场景，与详细的文本描述紧密对齐，在室内外和开放域场景中均优于基线方法。此外，HOLODECK 2.0提供了编辑能力，能够灵活适应人类反馈，支持布局优化和风格一致的对象编辑。最后，我们展示了HOLODECK 2.0在程序化游戏建模中的实际应用，以生成视觉丰富且沉浸的环境，从而提高游戏设计的效率。
英文摘要：
3D scene generation plays a crucial role in gaming, artistic creation, virtual reality, and many other domains. However, current 3D scene design still relies heavily on extensive manual effort from creators, and existing automated methods struggle to generate open-domain scenes or support flexible editing. To address those challenges, we introduce HOLODECK 2.0, an advanced vision-language-guided framework for 3D world generation with support for interactive scene editing based on human feedback. HOLODECK 2.0 can generate diverse and stylistically rich 3D scenes (e.g., realistic, cartoon, anime, and cyberpunk styles) that exhibit high semantic fidelity to fine-grained input descriptions, suitable for both indoor and open-domain environments. HOLODECK 2.0 leverages vision-language models (VLMs) to identify and parse the objects required in a scene and generates corresponding high-quality assets via state-of-the-art 3D generative models. Then, HOLODECK 2.0 iteratively applies spatial constraints derived from the VLMs to achieve semantically coherent and physically plausible layouts. Both human and model evaluations demonstrate that HOLODECK 2.0 effectively generates high-quality scenes closely aligned with detailed textual descriptions, consistently outperforming baselines across indoor and open-domain scenarios. Additionally, HOLODECK 2.0 provides editing capabilities that flexibly adapt to human feedback, supporting layout refinement and style-consistent object edits. Finally, we present a practical application of HOLODECK 2.0 in procedural game modeling to generate visually rich and immersive environments that can boost efficiency in game design Code is available at https://github.com/bzx20/Holodeck2.0.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
