arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.04543cs.CV
收藏

HyperDreamer:从单张图像进行超逼真3D内容生成与编辑

HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image

The Chinese University of Hong Kong(香港中文大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);S-Lab, NTU(南洋理工大学S-Lab实验室)

Tong Wu, Zhibing Li, Shuai Yang, Pan Zhang, Xinggang Pan, Jiaqi Wang, Dahua Lin, Ziwei Liu

AI总结:

本文提出HyperDreamer,从单图生成可360度查看、可渲染且可交互文本编辑的高分辨率超逼真3D模型。

摘要:

从单张图像创建3D内容是一项长期存在且极具需求的任务。近期进展引入了2D扩散先验,取得了合理的结果。然而,现有方法对于生成后的使用而言仍不够超逼真,因为用户无法从全范围查看、渲染和编辑所得3D内容。为应对这些挑战,我们提出HyperDreamer,它具有若干关键设计和吸引人的特性:1)可查看:具备高分辨率纹理的360度网格建模,能够从全范围观测点创建视觉上令人信服的3D模型。2)可渲染:结合细粒度语义分割和数据驱动先验作为引导,以学习材料合理的反照率、粗糙度和镜面属性,从而实现语义感知的任意材质估计。3)可编辑:对于生成的模型或用户自己的数据,用户只需点击几下即可交互式选择任意区域,并在文本引导下高效编辑纹理。大量实验证明了HyperDreamer在利用高分辨率纹理建模区域感知材质以及实现用户友好编辑方面的有效性。我们相信HyperDreamer有望推动3D内容创建,并在多个领域获得应用。

英文摘要:

3D content creation from a single image is a long-standing yet highly desirable task. Recent advances introduce 2D diffusion priors, yielding reasonable results. However, existing methods are not hyper-realistic enough for post-generation usage, as users cannot view, render and edit the resulting 3D content from a full range. To address these challenges, we introduce HyperDreamer with several key designs and appealing properties: 1) Viewable: 360 degree mesh modeling with high-resolution textures enables the creation of visually compelling 3D models from a full range of observation points. 2) Renderable: Fine-grained semantic segmentation and data-driven priors are incorporated as guidance to learn reasonable albedo, roughness, and specular properties of the materials, enabling semantic-aware arbitrary material estimation. 3) Editable: For a generated model or their own data, users can interactively select any region via a few clicks and efficiently edit the texture with text-based guidance. Extensive experiments demonstrate the effectiveness of HyperDreamer in modeling region-aware materials with high-resolution textures and enabling user-friendly editing. We believe that HyperDreamer holds promise for advancing 3D content creation and finding applications in various domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑