arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.09126cs.CV

DreamBarbie: 通过文本生成Barbie风格的3D人像

DreamBarbie: Text to Barbie-Style 3D Avatars

School of Intelligence Science and Technology, Nanjing University(智能科学与技术学院,南京大学);School of Computer Science, Peking University(计算机科学学院,北京大学)

Xiaokun Sun, Zhenyu Zhang, Ying Tai, Hao Tang, Zili Yi, Jian Yang

AI总结:

DreamBarbie通过文本生成Barbie风格的3D人像,结合表达性3D表示和专门优化模型,实现高保真度的可动画和模拟人像生成。

摘要:

为了将数字人融入日常生活,需要生成高质量、细粒度解耦的3D人像,以支持表达性动画和模拟能力,理想上可以从低成本的文本输入中生成。尽管通过利用2D生成先验知识,文本驱动的3D人像生成已取得显著进展,但现有方法仍难以同时满足所有要求。为解决这一挑战,我们提出了DreamBarbie,一种新的文本驱动框架,用于生成可动画化的3D人像,具有可分离的鞋子、配饰和模拟准备的衣物,真正捕捉到标志性的「Barbie娃娃」风格。框架的核心在于一个表达性的3D表示结合适当的建模约束。不同于先前的方法,我们使用G-Shell来统一建模封闭的组件(例如身体、鞋子)和非封闭的衣物。通过将边界重新表述为欧几里得场交集而不是流形测地线,我们提出了基于SDF的初始化和一个孔正则化损失,一起实现了100倍的速度提升和稳定的开放拓扑,而无需图像输入。这些解耦的3D表示随后通过专门针对每个领域的专家扩散模型进行优化,以确保高保真的输出。为了减轻在组合不同专家模型时出现的几何伪影和纹理冲突,我们进一步提出了几种有效的几何损失和策略。大量实验表明,DreamBarbie在穿着人类和服装生成方面均优于现有方法。我们的框架还使多样化的应用成为可能,包括服装搭配、编辑、表达性动画和物理模拟。项目页面:https://xiaokunsun.github.io/DreamBarbie.github.io/。

英文摘要:

To integrate digital humans into everyday life, there is a strong demand for generating high-quality, fine-grained disentangled 3D avatars that support expressive animation and simulation capabilities, ideally from low-cost textual inputs. Although text-driven 3D avatar generation has made significant progress by leveraging 2D generative priors, existing methods still struggle to fulfill all these requirements simultaneously. To address this challenge, we propose DreamBarbie, a novel text-driven framework for generating animatable 3D avatars with separable shoes, accessories, and simulation-ready garments, truly capturing the iconic ``Barbie doll'' aesthetic. The core of our framework lies in an expressive 3D representation combined with appropriate modeling constraints. Unlike prior methods, we use G-Shell to uniformly model watertight components (e.g., bodies, shoes) and non-watertight garments. By reformulating boundaries as Euclidean field intersections instead of manifold geodesics, we propose an SDF-based initialization and a hole regularization loss that together achieve a 100x speedup and stable open topology without image input. These disentangled 3D representations are then optimized by specialized expert diffusion models tailored to each domain, ensuring high-fidelity outputs. To mitigate geometric artifacts and texture conflicts when combining different expert models, we further propose several effective geometric losses and strategies. Extensive experiments demonstrate that DreamBarbie outperforms existing methods in both dressed human and outfit generation. Our framework further enables diverse applications, including apparel combination, editing, expressive animation, and physical simulation. Project page: https://xiaokunsun.github.io/DreamBarbie.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑