驯服文本到三维生成中分数蒸馏的模式坍缩问题
Taming Mode Collapse in Score Distillation for Text-to-3D Generation
AI总结：
本文揭示文本到三维生成中分数蒸馏的 Janus 伪影源于逐视图最大似然导致的模式坍缩，提出通过恢复变分目标中的熵项并推导出熵分数蒸馏（ESD），以无分类器引导的简单实现有效缓解视图不一致问题。
摘要：
尽管分数蒸馏在文本到三维生成中表现出色，这类技术却因视图不一致问题而饱受诟病，该问题也被称为“Janus”伪影，即生成物体在每个视图中都伪造出多个正面。虽然已有经验上有效的方法通过分数去偏或提示工程来应对这一问题，但用于解释和解决该问题的更严格视角仍然缺失。在本文中，我们揭示了现有基于分数蒸馏的文本到三维生成框架会退化为在每个视图上独立地寻求最大似然，从而遭受模式坍缩问题，在实践中表现为 Janus 伪影。为了驯服模式坍缩，我们通过在与渲染图像分布相对应的变分目标中重新建立熵项来改进分数蒸馏。最大化熵会鼓励生成三维资产中不同视图之间的多样性，从而缓解 Janus 问题。基于这一新目标，我们为三维分数蒸馏推导出一条新的更新规则，称为熵分数蒸馏（Entropic Score Distillation，ESD）。我们从理论上揭示，ESD 可以被简化并实现为仅在变分分数蒸馏上采用无分类器引导技巧。尽管实现方式简单得令人惊讶，我们的大量实验成功证明，ESD 可以成为分数蒸馏中 Janus 伪影的有效处理方法。
英文摘要：
Despite the remarkable performance of score distillation in text-to-3D generation, such techniques notoriously suffer from view inconsistency issues, also known as "Janus" artifact, where the generated objects fake each view with multiple front faces. Although empirically effective methods have approached this problem via score debiasing or prompt engineering, a more rigorous perspective to explain and tackle this problem remains elusive. In this paper, we reveal that the existing score distillation-based text-to-3D generation frameworks degenerate to maximal likelihood seeking on each view independently and thus suffer from the mode collapse problem, manifesting as the Janus artifact in practice. To tame mode collapse, we improve score distillation by re-establishing the entropy term in the corresponding variational objective, which is applied to the distribution of rendered images. Maximizing the entropy encourages diversity among different views in generated 3D assets, thereby mitigating the Janus problem. Based on this new objective, we derive a new update rule for 3D score distillation, dubbed Entropic Score Distillation (ESD). We theoretically reveal that ESD can be simplified and implemented by just adopting the classifier-free guidance trick upon variational score distillation. Although embarrassingly straightforward, our extensive experiments successfully demonstrate that ESD can be an effective treatment for Janus artifacts in score distillation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
