arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.14860cs.CV
收藏

DiffSurf:一种基于Transformer的扩散模型,用于生成和重建位姿中的3D表面

DiffSurf: A Transformer-based Diffusion Model for Generating and Reconstructing 3D Surfaces in Pose

National Institute of Advanced Industrial Science and Technology (AIST)(日本国立先进工业科学技术研究院(AIST))

Yusuke Yoshiyasu, Leyuan Sun

AI总结:

本文提出基于Transformer的去噪扩散模型DiffSurf,通过预测含噪3D表面顶点与法线来生成和重建多姿态3D表面,并能处理变形、体型变化及2D关键点拟合等下游任务,在3D人体基准上实现了更优的生成质量与近实时恢复精度。

摘要:

本文提出了DiffSurf,一种基于Transformer的去噪扩散模型,用于生成和重建3D表面。具体而言,我们设计了一种扩散Transformer架构,可从含噪声的3D表面顶点和法线中预测噪声。借助该架构,DiffSurf能够生成具有各种位姿和形状的3D表面,例如人体、手部、动物和人造物体。此外,DiffSurf具有通用性,能够处理多种3D下游任务,包括变形、体型变化以及将3D人体网格拟合到2D关键点。在3D人体模型基准上的实验结果表明,与以前的生成模型相比,DiffSurf能生成具有更大多样性和更高质量的形状。此外,应用于单图像3D人体网格恢复任务时,DiffSurf以接近实时的速率实现了与先前技术相当的准确度。

英文摘要:

This paper presents DiffSurf, a transformer-based denoising diffusion model for generating and reconstructing 3D surfaces. Specifically, we design a diffusion transformer architecture that predicts noise from noisy 3D surface vertices and normals. With this architecture, DiffSurf is able to generate 3D surfaces in various poses and shapes, such as human bodies, hands, animals and man-made objects. Further, DiffSurf is versatile in that it can address various 3D downstream tasks including morphing, body shape variation and 3D human mesh fitting to 2D keypoints. Experimental results on 3D human model benchmarks demonstrate that DiffSurf can generate shapes with greater diversity and higher quality than previous generative models. Furthermore, when applied to the task of single-image 3D human mesh recovery, DiffSurf achieves accuracy comparable to prior techniques at a near real-time rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑