DEFT：通过学习广义$h$-变换实现扩散模型的高效微调
DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised $h$-transform
AI总结：
针对现有扩散模型条件采样方法缺乏统一框架、训练成本高的问题，本研究基于Doob's h-变换提出DEFT方法，仅微调小型网络学习条件h-变换，在多基准任务上实现速度与性能的双重提升。
摘要：
基于去噪扩散过程的生成建模范式已成为逆问题中条件采样的主流候选方案。在许多实际应用中，我们通常能获取到经过大量成本训练的大规模无条件扩散模型，目标是利用这些模型来提升条件采样效果。近期大多数方法的动机都源于启发式思路，缺乏统一框架，模糊了各类方法之间的关联。此外，这些方法常存在超参数敏感性高、训练成本高昂，或是需要访问封闭API背后隐藏权重等问题。\n在本研究中，我们利用数学上已被充分研究的Doob's h-变换（杜布h-变换），将条件训练与采样过程统一起来。这一新视角使我们能将众多现有方法纳入同一框架体系下。在此框架基础上，我们提出了DEFT（Doob's h-transform Efficient FineTuning，杜布h-变换高效微调）——一种全新的条件生成方法，该方法仅微调一个极小的网络来快速学习条件$h$-变换，同时保持更大规模的无条件网络固定不变。\nDEFT的速度远快于现有基线方法，同时在各类线性与非线性基准测试中取得了当前最优性能。在图像重建任务上，我们实现了最高1.6倍的速度提升，同时在自然图像上具备最优的感知质量，在医学图像上具备最优的重建性能。此外，我们还在蛋白质基序支架构建任务上开展了初步实验，其表现优于重建引导方法。
英文摘要：
Generative modelling paradigms based on denoising diffusion processes have emerged as a leading candidate for conditional sampling in inverse problems. In many real-world applications, we often have access to large, expensively trained unconditional diffusion models, which we aim to exploit for improving conditional sampling. Most recent approaches are motivated heuristically and lack a unifying framework, obscuring connections between them. Further, they often suffer from issues such as being very sensitive to hyperparameters, being expensive to train or needing access to weights hidden behind a closed API. In this work, we unify conditional training and sampling using the mathematically well-understood Doob's h-transform. This new perspective allows us to unify many existing methods under a common umbrella. Under this framework, we propose DEFT (Doob's h-transform Efficient FineTuning), a new approach for conditional generation that simply fine-tunes a very small network to quickly learn the conditional $h$-transform, while keeping the larger unconditional network unchanged. DEFT is much faster than existing baselines while achieving state-of-the-art performance across a variety of linear and non-linear benchmarks. On image reconstruction tasks, we achieve speedups of up to 1.6$\times$, while having the best perceptual quality on natural images and reconstruction performance on medical images. Further, we also provide initial experiments on protein motif scaffolding and outperform reconstruction guidance methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
