Recently, diffusion models have been used to solve various inverse problems in an unsupervised manner with appropriate modifications to the sampling process. However, the current solvers, which recursively apply a reverse diffusion step followed by a projection-based measurement consistency step, often produce suboptimal results. By studying the generative sampling path, here we show that current solvers throw the sample path off the data manifold, and hence the error accumulates. To address this, we propose an additional correction term inspired by the manifold constraint, which can be used synergistically with the previous solvers to make the iterations close to the manifold. The proposed manifold constraint is straightforward to implement within a few lines of code, yet boosts the performance by a surprisingly large margin. With extensive experiments, we show that our method is superior to the previous methods both theoretically and empirically, producing promising results in many applications such as image inpainting, colorization, and sparse-view computed tomography. Code available https://github.com/HJ-harry/MCG_diffusion
Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint
ZJU-UIUC Institute(ZJU-UIUC联合学院);Zhejiang University(浙江大学);School of Information Science and Technology(信息科学与技术学院);Tsinghua University(清华大学);Institute for Physics of Intelligence(智能物理研究所);The University of Tokyo(东京大学)
EquiReg: Equivariance Regularized Diffusion for Inverse Problems
University of Alberta(阿尔伯塔大学);Alberta Machine, Intelligence Institute (Amii)(阿尔伯塔机器智能研究所);Neuroscience and Mental Health Institute, University of Alberta(阿尔伯塔大学神经科学与心理健康研究所);New York University(纽约大学);California Institute of Technology(加州理工学院);University of Oxford(牛津大学)
Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity
Center for Data Science, Peking University(北京大学数据科学中心);Zuse Institute Berlin(柏林泽尼茨研究所);Center for Machine Learning Research, Peking University(北京大学机器学习研究中心);LMAM and School of Mathematical Sciences, Peking University(北京大学数学科学学院)
Improving Dense Crowd Counting Convolutional Neural Networks using Inverse k-Nearest Neighbor Maps and Multiscale Upsampling
The Graduate Center of the City University of New York(纽约城市大学研究生中心);Borough of Manhattan Community College - CUNY(纽约城市大学曼哈顿社区学院);The City College of New York - CUNY(纽约城市大学纽约城市学院)
Towards a machine learning pipeline in reduced order modelling for inverse problems: neural networks for boundary parametrization, dimensionality reduction and solution manifold approximation