DreamBeast：通过部件感知知识迁移蒸馏生成3D奇幻动物
DreamBeast: Distilling 3D Fantastical Animals with Part-Aware Knowledge Transfer
AI总结：
针对现有SDS方法生成3D奇幻动物时部件语义理解不足的问题，提出DreamBeast方法，通过部件感知知识迁移将Stable Diffusion 3的部件知识融入3D表示，提升生成质量并降低计算开销。
摘要：
我们提出DreamBeast，这是一种基于分数蒸馏采样（SDS）的新型方法，用于生成由不同部件组成的奇幻3D动物资产。现有SDS方法由于文本到图像扩散模型对部件级语义的理解有限，往往难以完成该生成任务。尽管近期的扩散模型（如Stable Diffusion 3）展现出更好的部件级理解能力，但它们速度极慢，且存在单视图扩散模型共有的其他常见问题。DreamBeast通过一种新型部件感知知识迁移机制克服了这一局限。对于每个生成的资产，我们高效地将Stable Diffusion 3模型中的部件级知识提取到3D部件亲和隐式表示中。这使我们能够从任意相机视角即时生成部件亲和图，随后在SDS过程中用这些图调制多视图扩散模型的引导，从而创建奇幻动物的3D资产。大量定量和定性评估表明，DreamBeast显著提升了具有用户指定部件组合的3D生物生成质量，同时降低了计算开销。
英文摘要：
We present DreamBeast, a novel method based on score distillation sampling (SDS) for generating fantastical 3D animal assets composed of distinct parts. Existing SDS methods often struggle with this generation task due to a limited understanding of part-level semantics in text-to-image diffusion models. While recent diffusion models, such as Stable Diffusion 3, demonstrate a better part-level understanding, they are prohibitively slow and exhibit other common problems associated with single-view diffusion models. DreamBeast overcomes this limitation through a novel part-aware knowledge transfer mechanism. For each generated asset, we efficiently extract part-level knowledge from the Stable Diffusion 3 model into a 3D Part-Affinity implicit representation. This enables us to instantly generate Part-Affinity maps from arbitrary camera views, which we then use to modulate the guidance of a multi-view diffusion model during SDS to create 3D assets of fantastical animals. DreamBeast significantly enhances the quality of generated 3D creatures with user-specified part compositions while reducing computational overhead, as demonstrated by extensive quantitative and qualitative evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
