arXiv 2412.000912024-12-06cs.CVcs.AIcs.GR

用于可控 3D 场景生成的 Graph Canvas

Graph Canvas for Controllable 3D Scene Generation

Beijing University of Technology（北京工业大学）；East China University of Science（华东理工大学）；Shandong University（山东大学）；Tsinghua University（清华大学）；Rutgers University（罗格斯大学）；University of Washington（华盛顿大学）；University of Copenhagen（哥本哈根大学）

Libin Liu, Shen Chen, Sen Jia, Jingzhe Shi, Zhongyu Jiang, Can Jin, Wu Zongkai, Jenq-Neng Hwang, Lei Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出 GraphCanvas3D 框架，利用上下文学习和分层图驱动描述实现无需重训的可控 3D/4D 场景生成与动态调整。

摘要：

空间智能是与物理世界交互的 AI 系统的基础，尤其体现在 3D 场景生成和空间理解方面。当前的 3D 场景生成方法往往严重依赖预定义数据集，难以动态适应不断变化的空间关系。本文提出 GraphCanvas3D，这是一个可编程、可扩展且适应性强的框架，用于可控 3D 场景生成。借助上下文学习，GraphCanvas3D 无需重新训练即可实现动态适应，支持灵活、可定制的场景创建。该框架采用分层的、图驱动的场景描述，将空间元素表示为图节点，并在 3D 环境中的对象之间建立连贯关系。传统方法在适应性上受到限制，修改时通常需要预定义输入掩码或重新训练；与之不同，GraphCanvas3D 支持即时、无缝的对象操控和场景调整。此外，GraphCanvas3D 支持 4D 场景生成，引入时间动态以建模随时间发生的变化。实验结果和用户研究表明，GraphCanvas3D 提升了场景生成的可用性、灵活性和适应性。代码和模型可在项目网站获取：https://github.com/ILGLJ/Graph-Canvas。

英文摘要：

Spatial intelligence is foundational to AI systems that interact with the physical world, particularly in 3D scene generation and spatial comprehension. Current methodologies for 3D scene generation often rely heavily on predefined datasets, and struggle to adapt dynamically to changing spatial relationships. In this paper, we introduce GraphCanvas3D, a programmable, extensible, and adaptable framework for controllable 3D scene generation. Leveraging in-context learning, GraphCanvas3D enables dynamic adaptability without the need for retraining, supporting flexible and customizable scene creation. Our framework employs hierarchical, graph-driven scene descriptions, representing spatial elements as graph nodes and establishing coherent relationships among objects in 3D environments. Unlike conventional approaches, which are constrained in adaptability and often require predefined input masks or retraining for modifications, GraphCanvas3D allows for seamless object manipulation and scene adjustments on the fly. Additionally, GraphCanvas3D supports 4D scene generation, incorporating temporal dynamics to model changes over time. Experimental results and user studies demonstrate that GraphCanvas3D enhances usability, flexibility, and adaptability for scene generation. Our code and models are available on the project website: https://github.com/ILGLJ/Graph-Canvas.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。