arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.10772cs.CV

HOG-Layout:通过视觉-语言模型实现层次化3D场景生成、优化与编辑

HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

The Hong Kong Polytechnic University(香港理工大学);Beijing Institute of Technology(北京理工大学)

Haiyan Jiang, Deyu Zhang, Dongdong Weng, Weitao Song, Henry Been-Lirn Duh

AI总结:

HOG-Layout利用大语言模型和视觉-语言模型实现文本驱动的层次化3D场景生成、优化与实时编辑,通过检索增强生成技术提升语义一致性,结合优化模块增强物理一致性,实现高效场景编辑。

摘要:

3D布局生成和编辑在具身AI和沉浸式VR交互中起着关键作用。然而,手动创建需要大量劳动,而数据驱动生成往往缺乏多样性。大模型的出现为3D场景合成带来了新可能。我们提出了HOG-Layout,该方法利用大语言模型(LLMs)和视觉-语言模型(VLMs)实现文本驱动的层次化场景生成、优化和实时场景编辑。HOG-Layout通过检索增强生成(RAG)技术提高场景语义一致性和合理性,结合优化模块增强物理一致性,并采用层次化表示以提升推理和优化效率,实现实时编辑。实验结果表明,HOG-Layout生成的环境相比现有基线更为合理,同时支持快速直观的场景编辑。

英文摘要:

3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction. However, manual creation requires tedious labor, while data-driven generation often lacks diversity. The emergence of large models introduces new possibilities for 3D scene synthesis. We present HOG-Layout that enables text-driven hierarchical scene generation, optimization and real-time scene editing with large language models (LLMs) and vision-language models (VLMs). HOG-Layout improves scene semantic consistency and plausibility through retrieval-augmented generation (RAG) technology, incorporates an optimization module to enhance physical consistency, and adopts a hierarchical representation to enhance inference and optimization, achieving real-time editing. Experimental results demonstrate that HOG-Layout produces more reasonable environments compared with existing baselines, while supporting fast and intuitive scene editing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑