arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.18780cs.CV
收藏

基于分数蒸馏采样的文本到3D生成的定量评估

A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D

AWS AI Labs(亚马逊云科技人工智能实验室)

Xiaohan Fei, Chethan Parameshwara, Jiawei Mo, Xiaolong Li, Ashwin Swaminathan, CJ Taylor, Paolo Favaro, Stefano Soatto

AI总结:

针对文本到3D中SDS导致的Janus、错位和几何不准确问题,本文提出经人类评分交叉验证的定量指标,并据此设计高效基线模型以消除伪影并达到最优指标。

摘要:

借助在用于图像生成的预训练扩散模型上使用分数蒸馏采样(SDS)方法,根据文本提示创建3D内容的生成模型已取得长足进步。然而,SDS方法也是若干伪影的来源,例如Janus问题、文本提示与生成3D模型之间的错位,以及3D模型不准确。现有方法严重依赖通过目视检查有限样本来对这些伪影进行定性评估,而在本工作中,我们提出更客观的定量评估指标,并通过人类评分进行交叉验证,同时展示了对SDS技术失败案例的分析。我们设计了一种新颖且计算高效的基线模型,在解决上述所有伪影的同时,在所提出的指标上达到最先进性能,由此证明了该分析的有效性。

英文摘要:

The development of generative models that create 3D content from a text prompt has made considerable strides thanks to the use of the score distillation sampling (SDS) method on pre-trained diffusion models for image generation. However, the SDS method is also the source of several artifacts, such as the Janus problem, the misalignment between the text prompt and the generated 3D model, and 3D model inaccuracies. While existing methods heavily rely on the qualitative assessment of these artifacts through visual inspection of a limited set of samples, in this work we propose more objective quantitative evaluation metrics, which we cross-validate via human ratings, and show analysis of the failure cases of the SDS technique. We demonstrate the effectiveness of this analysis by designing a novel computationally efficient baseline model that achieves state-of-the-art performance on the proposed metrics while addressing all the above-mentioned artifacts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑