Diff2Flow:通过扩散模型对齐训练流匹配模型
Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
AI总结:
本文提出 Diff2Flow 框架,通过时间步重缩放、插值器对齐和速度场推导,将预训练扩散模型高效迁移到流匹配微调,在参数高效约束下优于现有方法。
摘要:
扩散模型凭借高保真输出革新了生成任务,然而流匹配(FM)提供了更快的推理速度和经验性能提升。但目前的基础流匹配模型在微调时计算成本过高,而像 Stable Diffusion 这样的扩散模型则受益于高效的架构和生态支持。本文解决了将预训练扩散模型中的知识高效迁移到流匹配这一关键挑战。我们提出了 Diff2Flow,这是一个新颖的框架,通过重新缩放时间步、对齐插值器,并从扩散预测中推导出与流匹配兼容的速度场,系统性地桥接了扩散和流匹配两种范式。这种对齐使得对扩散先验进行直接且高效的流匹配微调成为可能,且不产生额外的计算开销。我们的实验表明,Diff2Flow 在参数高效约束下尤其优于朴素的流匹配和扩散微调,同时在各种下游任务中与最先进的方法相比取得了更优或具有竞争力的性能。我们将在 https://github.com/CompVis/diff2flow 发布代码。
英文摘要:
Diffusion models have revolutionized generative tasks through high-fidelity outputs, yet flow matching (FM) offers faster inference and empirical performance gains. However, current foundation FM models are computationally prohibitive for finetuning, while diffusion models like Stable Diffusion benefit from efficient architectures and ecosystem support. This work addresses the critical challenge of efficiently transferring knowledge from pre-trained diffusion models to flow matching. We propose Diff2Flow, a novel framework that systematically bridges diffusion and FM paradigms by rescaling timesteps, aligning interpolants, and deriving FM-compatible velocity fields from diffusion predictions. This alignment enables direct and efficient FM finetuning of diffusion priors with no extra computation overhead. Our experiments demonstrate that Diff2Flow outperforms naïve FM and diffusion finetuning particularly under parameter-efficient constraints, while achieving superior or competitive performance across diverse downstream tasks compared to state-of-the-art methods. We will release our code at https://github.com/CompVis/diff2flow.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。