扰动与求解：潜空间扩散逆问题的高效学习算子条件化
Perturb-and-Solve: Efficient Learned-Operator Conditioning for Latent Diffusion Inverse Problems
AI总结：
针对潜空间扩散逆问题，提出PASEO方法，用小型学习网络实现高效算子条件化，避免反向传播，在超分辨率、去模糊和修复中速度提升9倍、内存减少34%。
AI中文摘要：
潜空间扩散模型作为图像恢复中求解逆问题（如去模糊、修复和超分辨率）的强大先验。现有方法在通用性和效率之间存在权衡。限制在固定退化算子集合内的求解器速度快且高效，而支持任意退化算子的方法速度慢且需要穿过扩散网络进行梯度计算。为打破这一瓶颈，我们提出了PASEO（扰动与求解的高效算子条件化），该方法使用一个小型（1M参数）学习网络在潜空间中对扩散模型的预测进行退化处理。PASEO支持学习到的退化算子，而无需通过扩散网络进行反向传播。我们通过结合扩散模型的预测与观测图像，从近似后验中高效采样重建结果。具体做法是添加噪声并基于学习网络的局部线性近似求解线性方程，无需构建或求逆大型协方差矩阵。在FFHQ和COCO数据集上的超分辨率、去模糊和修复任务中，PASEO在保持强感知质量的同时，运行速度比测试的基线方法快最多9倍，峰值内存使用减少最多34%，且模型评估次数相同或更少。
英文摘要：
Latent diffusion models serve as powerful priors for solving inverse problems in image restoration, such as deblurring, inpainting, and super-resolution. Current methods have a trade-off between generality and efficiency. Solvers that are restricted to a fixed set of degradation operators are fast and efficient. Methods that support arbitrary degradation operators are slow and require gradients through the diffusion network. To break this bottleneck, we introduce PASEO (Perturb-And-Solve for Efficient Operator conditioning), a method that uses a small (1M parameters) learned network to degrade diffusion model predictions in latent space. PASEO supports learned degradation operators without back-propagating through the diffusion network. We efficiently sample reconstructions from an approximate posterior by combining the diffusion model's prediction with the observed image. We do this by adding noise and solving linear equations based on a local linear approximation of the learned network, without building or inverting large covariance matrices. Across super-resolution, deblurring, and inpainting on FFHQ and COCO, PASEO achieves strong perceptual quality while running up to 9x faster and using up to 34% less peak memory than the tested baselines, with the same or fewer model evaluations.
