arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.11535cs.CVcs.AI
收藏

利用特定对象知识先验从单张图像生成高质量3D内容

High-Quality 3D Creation from A Single Image Using Subject-Specific Knowledge Prior

Peking University(北京大学);Beijing Normal University(北京师范大学);Microsoft Research Asia(微软亚洲研究院)

Nan Huang, Ting Zhang, Yuhui Yuan, Dong Chen, Shanghang Zhang

AI总结:

针对机器人领域多样化3D数据稀缺的问题,提出两阶段方法,利用特定对象几何与纹理先验(含阴影模式感知先验)提升NeRF优化效果,生成高质量3D模型,显著优于依赖通用扩散先验的现有方法。

摘要:

本文针对机器人领域中多样化3D数据稀缺这一关键瓶颈,提出了一种从单张图像生成高质量3D模型的新颖两阶段方法。该方法的动机是高效扩展3D资产创建,尤其是在机器人数据集方面——当前其对象类型多样性与通用图像数据集相比仍有限。与以往主要依赖通用扩散先验(这类方法常难以与参考图像对齐)的方法不同,我们的方法利用特定对象先验知识。通过在几何和纹理中融入特定对象先验,我们确保生成的3D内容与参考对象精确对齐。具体而言,我们将阴影模式感知先验引入NeRF优化过程,增强几何结构并细化粗输出的纹理以实现更优质量。大量实验表明,我们的方法显著优于现有方法。

英文摘要:

In this paper, we address the critical bottleneck in robotics caused by the scarcity of diverse 3D data by presenting a novel two-stage approach for generating high-quality 3D models from a single image. This method is motivated by the need to efficiently expand 3D asset creation, particularly for robotics datasets, where the variety of object types is currently limited compared to general image datasets. Unlike previous methods that primarily rely on general diffusion priors, which often struggle to align with the reference image, our approach leverages subject-specific prior knowledge. By incorporating subject-specific priors in both geometry and texture, we ensure precise alignment between the generated 3D content and the reference object. Specifically, we introduce a shading mode-aware prior into the NeRF optimization process, enhancing the geometry and refining texture in the coarse outputs to achieve superior quality. Extensive experiments demonstrate that our method significantly outperforms prior approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑