arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.18035cs.LGcs.CV
收藏

双向一致性模型

Bidirectional Consistency Models

University of Cambridge(剑桥大学)

Liangchen Li, Jiajun He

AI总结:

提出双向一致性模型(BCM),通过单个神经网络统一实现沿概率流常微分方程的前向生成与反向反演,支持单步及多步操作,降低训练成本并提升下游任务效果。

摘要:

扩散模型(DMs)能够通过迭代去噪随机向量来生成极高质量的样本,这一过程对应于沿着概率流常微分方程(PF ODE)移动。有趣的是,DMs 也可以通过沿着 PF ODE 反向移动来将输入图像反演为噪声,这是插值和图像编辑等下游任务的关键操作。然而,该过程的迭代特性限制了其速度,阻碍了其更广泛的应用。最近,一致性模型(CMs)通过近似 PF ODE 的积分来解决这一挑战,大幅减少了迭代次数。但是,缺乏显式的 ODE 求解器使反演过程变得复杂。为了解决这个问题,我们引入了 Bidirectional Consistency Model(BCM,双向一致性模型),它学习单个神经网络,使得能够沿着 PF ODE 进行前向和反向遍历,从而在一个框架内高效地统一了生成和反演任务。我们可以从头训练 BCM,或者使用预训练的一致性模型对其进行微调,这降低了训练成本并提高了可扩展性。我们证明,BCM 能够实现单步生成和反演,同时也允许使用额外的步骤来提高生成质量或减少重建误差。我们进一步展示了 BCM 在插值和图像修复等下游任务中的能力。我们的代码和权重可在 https://github.com/Mosasaur5526/BCM-iCT-torch 获取。

英文摘要:

Diffusion models (DMs) are capable of generating remarkably high-quality samples by iteratively denoising a random vector, a process that corresponds to moving along the probability flow ordinary differential equation (PF ODE). Interestingly, DMs can also invert an input image to noise by moving backward along the PF ODE, a key operation for downstream tasks such as interpolation and image editing. However, the iterative nature of this process restricts its speed, hindering its broader application. Recently, Consistency Models (CMs) have emerged to address this challenge by approximating the integral of the PF ODE, largely reducing the number of iterations. Yet, the absence of an explicit ODE solver complicates the inversion process. To resolve this, we introduce Bidirectional Consistency Model (BCM), which learns a single neural network that enables both forward and backward traversal along the PF ODE, efficiently unifying generation and inversion tasks within one framework. We can train BCM from scratch or tune it using a pretrained consistency model, which reduces the training cost and increases scalability. We demonstrate that BCM enables one-step generation and inversion while also allowing the use of additional steps to enhance generation quality or reduce reconstruction error. We further showcase BCM's capability in downstream tasks, such as interpolation and inpainting. Our code and weights are available at https://github.com/Mosasaur5526/BCM-iCT-torch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑