arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.01202cs.CV
收藏

SinGRAV:从单个自然场景学习生成辐射体积

SinGRAV: Learning a Generative Radiance Volume from a Single Natural Scene

Shandong University(山东大学);Tencent AI Lab(腾讯人工智能实验室);Peking University(北京大学)

Yujie Wang, Xuelin Chen, Baoquan Chen

摘要:

我们提出了一个一般的自然场景的三维生成模型。缺乏必要的卷的3D数据表征的目标场景,我们建议从一个单一的场景学习。我们的关键见解是,一个自然场景往往包含多个组成部分,其几何形状,纹理和空间布局遵循一些明确的模式,但仍然表现出丰富的变化在同一场景内的不同区域。这意味着将生成模型的学习局限于大量的局部区域。因此,我们利用一个多尺度卷积网络,它具有空间局部性偏差的性质,从一个场景中的多个尺度的局部区域的统计信息中学习。与现有方法相比,我们的学习设置绕过了从许多同质3D场景中收集数据以学习常见特征的需要。我们硬币我们的方法SinGRAV,从一个单一的自然场景学习生成辐射体积。我们展示了SinGRAV从单个场景生成合理和多样化变化的能力,SinGRAV超过最先进的生成神经场景方法的优点,以及SinGRAV在各种应用中的多功能性,包括3D场景编辑,合成和动画。将公布代码和数据,以方便进一步的研究。

英文摘要:

We present a 3D generative model for general natural scenes. Lacking necessary volumes of 3D data characterizing the target scene, we propose to learn from a single scene. Our key insight is that a natural scene often contains multiple constituents whose geometry, texture, and spatial arrangements follow some clear patterns, but still exhibit rich variations over different regions within the same scene. This suggests localizing the learning of a generative model on substantial local regions. Hence, we exploit a multi-scale convolutional network, which possesses the spatial locality bias in nature, to learn from the statistics of local regions at multiple scales within a single scene. In contrast to existing methods, our learning setup bypasses the need to collect data from many homogeneous 3D scenes for learning common features. We coin our method SinGRAV, for learning a Generative RAdiance Volume from a Single natural scene. We demonstrate the ability of SinGRAV in generating plausible and diverse variations from a single scene, the merits of SinGRAV over state-of-the-art generative neural scene methods, as well as the versatility of SinGRAV by its use in a variety of applications, spanning 3D scene editing, composition, and animation. Code and data will be released to facilitate further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑