arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.14700cs.CV
收藏

用改进的AR模型学习通用三维形状生成

Learning Versatile 3D Shape Generation with Improved AR Models

Fudan University(复旦大学);Google(谷歌);Tencent Youtu Lab(腾讯优图实验室)

Simian Luo, Xuelin Qian, Yanwei Fu, Yinda Zhang, Ying Tai, Zhenyu Zhang, Chengjie Wang, Xiangyang Xue

摘要:

自回归(AR)模型通过对网格空间中的联合分布进行建模,在二维图像生成中取得了令人印象深刻的结果。虽然这种方法已经扩展到3D域的强大的形状生成,它仍然有两个限制:昂贵的计算体积网格和模糊的自回归顺序沿网格尺寸。为了克服这些限制,我们提出了改进的自回归模型(ImAM)的三维形状生成,它适用于离散表示学习的基础上的一个潜在的矢量,而不是体积网格。我们的方法不仅降低了计算成本,而且通过以更易处理的顺序学习联合分布来保留基本的几何细节。此外,由于模型架构的简单性,我们可以通过连接各种条件输入(如点云、类别、图像和文本),自然地将其从无条件生成扩展到有条件生成。大量的实验表明,ImAM可以合成多种类别的多样和忠实的形状,达到最先进的性能。

英文摘要:

Auto-Regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. While this approach has been extended to the 3D domain for powerful shape generation, it still has two limitations: expensive computations on volumetric grids and ambiguous auto-regressive order along grid dimensions. To overcome these limitations, we propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids. Our approach not only reduces computational costs but also preserves essential geometric details by learning the joint distribution in a more tractable order. Moreover, thanks to the simplicity of our model architecture, we can naturally extend it from unconditional to conditional generation by concatenating various conditioning inputs, such as point clouds, categories, images, and texts. Extensive experiments demonstrate that ImAM can synthesize diverse and faithful shapes of multiple categories, achieving state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑