arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.07877stat.MLcs.LG
收藏

Score-Optimal Diffusion Schedules(得分最优扩散调度)

Score-Optimal Diffusion Schedules

University of Oxford(牛津大学)

Christopher Williams, Andrew Campbell, Arnaud Doucet, Saifuddin Syed

AI总结:

针对去噪扩散模型采样离散化调度缺乏通用选择方法的问题,提出基于传输功代价函数的自适应最优调度算法,仅需估计Stein得分,无需调优,可复现人工搜索的高性能调度并取得有竞争力的FID分数。

摘要:

去噪扩散模型(DDMs)为从高维数据分布中采样提供了灵活的框架。DDMs通过向数据中逐步注入噪声,生成一条在参考高斯分布与数据分布之间插值的概率分布路径。为对采样过程进行数值模拟,必须选择一条从参考分布回溯到干净数据的离散化调度。合适的离散化调度对于获得高质量样本至关重要。然而,除了手工设计的启发式方法外,选择该调度的通用方法仍未找到。本文提出了一种新算法,可针对我们推导的代价函数自适应选择最优离散化调度。我们的代价函数衡量模拟过程将样本从扩散路径上的一个点传输到下一个点所做的功。我们的方法无需超参数调优,且能适应扩散路径的动力学特性与几何结构。我们的算法仅涉及对估计的Stein得分的计算,因此在推理时以及训练过程中均可扩展到现有预训练模型。我们发现,我们学习到的调度能够复现此前仅通过人工搜索发现的高性能调度,并且在图像数据集上取得了具有竞争力的FID分数。

英文摘要:

Denoising diffusion models (DDMs) offer a flexible framework for sampling from high dimensional data distributions. DDMs generate a path of probability distributions interpolating between a reference Gaussian distribution and a data distribution by incrementally injecting noise into the data. To numerically simulate the sampling process, a discretisation schedule from the reference back towards clean data must be chosen. An appropriate discretisation schedule is crucial to obtain high quality samples. However, beyond hand crafted heuristics, a general method for choosing this schedule remains elusive. This paper presents a novel algorithm for adaptively selecting an optimal discretisation schedule with respect to a cost that we derive. Our cost measures the work done by the simulation procedure to transport samples from one point in the diffusion path to the next. Our method does not require hyperparameter tuning and adapts to the dynamics and geometry of the diffusion path. Our algorithm only involves the evaluation of the estimated Stein score, making it scalable to existing pre-trained models at inference time and online during training. We find that our learned schedule recovers performant schedules previously only discovered through manual search and obtains competitive FID scores on image datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑