Diffusion Inverse Solvers (DIS) are designed to sample from the conditional distribution $p_θ(X_0|y)$, with a predefined diffusion model $p_θ(X_0)$, an operator $f(\cdot)$, and a measurement $y=f(x'_0)$ derived from an unknown image $x'_0$. Existing DIS estimate the conditional score function by evaluating $f(\cdot)$ with an approximated posterior sample drawn from $p_θ(X_0|X_t)$. However, most prior approximations rely on the posterior means, which may not lie in the support of the image distribution, thereby potentially diverge from the appearance of genuine images. Such out-of-support samples may significantly degrade the performance of the operator $f(\cdot)$, particularly when it is a neural network. In this paper, we introduces a novel approach for posterior approximation that guarantees to generate valid samples within the support of the image distribution, and also enhances the compatibility with neural network-based operators $f(\cdot)$. We first demonstrate that the solution of the Probability Flow Ordinary Differential Equation (PF-ODE) with an initial value $x_t$ yields an effective posterior sample $p_θ(X_0|X_t=x_t)$. Based on this observation, we adopt the Consistency Model (CM), which is distilled from PF-ODE, for posterior sampling. Furthermore, we design a novel family of DIS using only CM. Through extensive experiments, we show that our proposed method for posterior sample approximation substantially enhance the effectiveness of DIS for neural network operators $f(\cdot)$ (e.g., in semantic segmentation). Additionally, our experiments demonstrate the effectiveness of the new CM-based inversion techniques. The source code is provided in the supplementary material.
Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint
ZJU-UIUC Institute(ZJU-UIUC联合学院);Zhejiang University(浙江大学);School of Information Science and Technology(信息科学与技术学院);Tsinghua University(清华大学);Institute for Physics of Intelligence(智能物理研究所);The University of Tokyo(东京大学)
LATINO-PRO: LAtent consisTency INverse sOlver with PRompt Optimization
CNRS(法国国家科学研究中心);Univ. Bordeaux(波尔多大学);INRIA(法国国家信息与自动化研究所);Bordeaux INP(波尔多国立高等工程技术学院);Heriot-Watt University(赫瑞瓦特大学);Maxwell Institute for Mathematical Sciences(麦克斯韦数学科学研究所)
LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Benchmarking Diffusion Annealing-Based Bayesian Inverse Problem Solvers
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin(奥登计算工程与科学研究所,德克萨斯大学奥斯汀分校);Dept of Biomedical Engineering, The University of Texas at Austin(生物医学工程系,德克萨斯大学奥斯汀分校)
Measured-Subspace Consistency: A Plug-and-Play Operator for Diffusion Posterior Sampling in Accelerated MRI Reconstruction
Interdisciplinary Program in Cancer Biology, Seoul National University College of Medicine(首尔国立大学医学院癌症生物学跨学科项目);Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital(首尔国立大学医学院放射科,首尔国立大学医院);Healthcare AI Research Institute, Seoul National University Hospital(首尔国立大学医院医疗人工智能研究机构)