更大并不总是更好：潜在扩散模型的缩放特性
Bigger is not Always Better: Scaling Properties of Latent Diffusion Models
AI总结：
本文实证研究文本到图像 LDM 的模型规模与采样效率关系，发现固定推理预算下小模型常优于大模型，并在多种采样器、下游任务、蒸馏模型及训练算力比较中验证其普适性。
摘要：
我们研究潜在扩散模型（LDM）的缩放特性，重点关注其采样效率。尽管改进的网络架构和推理算法已被证明能够有效提升扩散模型的采样效率，但模型规模这一决定采样效率的关键因素尚未得到充分检验。通过对成熟的文本到图像扩散模型进行实证分析，我们深入研究了模型规模如何在不同采样步数下影响采样效率。研究结果揭示了一个令人惊讶的趋势：在给定推理预算下运行时，较小模型在生成高质量结果方面往往优于较大的同类模型。此外，我们通过应用多种扩散采样器、探索不同下游任务、评估蒸馏后模型，以及比较相对于训练计算量的性能，扩展了研究以证明这些发现的普适性。这些发现为制定 LDM 缩放策略开辟了新途径，可用于在有限推理预算内增强生成能力。
英文摘要：
We study the scaling properties of latent diffusion models (LDMs) with an emphasis on their sampling efficiency. While improved network architecture and inference algorithms have shown to effectively boost sampling efficiency of diffusion models, the role of model size -- a critical determinant of sampling efficiency -- has not been thoroughly examined. Through empirical analysis of established text-to-image diffusion models, we conduct an in-depth investigation into how model size influences sampling efficiency across varying sampling steps. Our findings unveil a surprising trend: when operating under a given inference budget, smaller models frequently outperform their larger equivalents in generating high-quality results. Moreover, we extend our study to demonstrate the generalizability of the these findings by applying various diffusion samplers, exploring diverse downstream tasks, evaluating post-distilled models, as well as comparing performance relative to training compute. These findings open up new pathways for the development of LDM scaling strategies which can be employed to enhance generative capabilities within limited inference budgets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
