arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.12063cs.CVcs.LG
收藏

Consistency Model 是 Diffusion Inverse Solvers 的一种有效后验样本近似

Consistency Model is an Effective Posterior Sample Approximation for Diffusion Inverse Solvers

Tsinghua University(清华大学);Sensetime Research(商汤科技研究院);Kuaishou Technology(快手科技);Chinese Academy of Sciences(中国科学院)

Tongda Xu, Ziran Zhu, Jian Li, Dailan He, Yuanyuan Wang, Ming Sun, Ling Li, Hongwei Qin, Yan Wang, Jingjing Liu, Ya-Qin Zhang

AI总结:

针对现有扩散逆求解器因依赖后验均值导致样本偏离图像分布支撑集的问题,提出利用从 PF-ODE 蒸馏的 Consistency Model 生成有效后验样本,显著提升了基于神经网络算子的逆求解性能。

摘要:

Diffusion Inverse Solvers (DIS) 旨在从条件分布 $p_θ(X_0|y)$ 中采样,其包含预定义的扩散模型 $p_θ(X_0)$、算子 $f(\cdot)$ 以及源自未知图像 $x'_0$ 的测量值 $y=f(x'_0)$。现有 DIS 通过利用从 $p_θ(X_0|X_t)$ 中抽取的近似后验样本评估 $f(\cdot)$ 来估计条件得分函数。然而,多数先前的近似依赖后验均值,这可能不在图像分布的支撑集中,从而可能偏离真实图像的外观。这种支撑集外的样本可能显著降低算子 $f(\cdot)$ 的性能,尤其是当其为神经网络时。本文引入一种新颖的后验近似方法,保证生成位于图像分布支撑集内的有效样本,并增强与基于神经网络的算子 $f(\cdot)$ 的兼容性。我们首先证明,以 $x_t$ 为初始值的 Probability Flow Ordinary Differential Equation (PF-ODE) 的解可产生有效的后验样本 $p_θ(X_0|X_t=x_t)$。基于此观察,我们采用从 PF-ODE 蒸馏出的 Consistency Model (CM) 进行后验采样。此外,我们设计了一类仅使用 CM 的新型 DIS 家族。通过大量实验,我们表明所提出的后验样本近似方法显著提升了 DIS 在神经网络算子 $f(\cdot)$(如语义分割)中的有效性。另外,实验也证明了新的基于 CM 的反演技术的有效性。源代码提供在补充材料中。

英文摘要:

Diffusion Inverse Solvers (DIS) are designed to sample from the conditional distribution $p_θ(X_0|y)$, with a predefined diffusion model $p_θ(X_0)$, an operator $f(\cdot)$, and a measurement $y=f(x'_0)$ derived from an unknown image $x'_0$. Existing DIS estimate the conditional score function by evaluating $f(\cdot)$ with an approximated posterior sample drawn from $p_θ(X_0|X_t)$. However, most prior approximations rely on the posterior means, which may not lie in the support of the image distribution, thereby potentially diverge from the appearance of genuine images. Such out-of-support samples may significantly degrade the performance of the operator $f(\cdot)$, particularly when it is a neural network. In this paper, we introduces a novel approach for posterior approximation that guarantees to generate valid samples within the support of the image distribution, and also enhances the compatibility with neural network-based operators $f(\cdot)$. We first demonstrate that the solution of the Probability Flow Ordinary Differential Equation (PF-ODE) with an initial value $x_t$ yields an effective posterior sample $p_θ(X_0|X_t=x_t)$. Based on this observation, we adopt the Consistency Model (CM), which is distilled from PF-ODE, for posterior sampling. Furthermore, we design a novel family of DIS using only CM. Through extensive experiments, we show that our proposed method for posterior sample approximation substantially enhance the effectiveness of DIS for neural network operators $f(\cdot)$ (e.g., in semantic segmentation). Additionally, our experiments demonstrate the effectiveness of the new CM-based inversion techniques. The source code is provided in the supplementary material.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑