arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.12100cs.CVcs.LGstat.ML
收藏

Boomerang:使用扩散模型在图像流形上进行局部采样

Boomerang: Local sampling on image manifolds using diffusion models

Rice University(莱斯大学)

Lorenzo Luzi, Paul M Mayer, Josue Casco-Rodriguez, Ali Siahkoohi, Richard G. Baraniuk

AI总结:

Boomerang通过向图像添加噪声并部分反向扩散,在图像流形上生成相似但不同的局部样本,支持隐私保护、数据增强和分辨率增强等应用。

摘要:

扩散模型的推理阶段可以看作运行一个反向时间扩散随机微分方程,其中来自高斯潜变量分布的样本被转换为通常位于低维流形(例如图像流形)上的目标分布的样本。初始潜空间与图像流形之间的中间值可以被解释为带噪图像,噪声量由前向扩散过程的噪声调度决定。我们利用这一解释提出了Boomerang,一种用于图像流形局部采样的方法。正如其名所示,Boomerang局部采样涉及向输入图像添加噪声,使其更接近潜空间,然后通过部分反向扩散过程将其映射回图像流形。因此,Boomerang在流形上生成与原始输入图像“相似”但不相同的图像。我们可以通过调整添加的噪声量来控制生成图像与原始图像的接近程度。此外,由于Boomerang中反向扩散过程的随机性,生成的图像表现出一定程度的随机性,使我们能够从流形中获得局部样本而不会遇到重复。Boomerang提供了与任何预训练扩散模型(如Stable Diffusion)无缝协作的灵活性,无需对反向扩散过程进行任何调整。我们展示了Boomerang的三个应用。首先,我们提供了一个构建具有可控匿名程度的隐私保护数据集的框架。其次,我们表明使用Boomerang进行数据增强可以提高泛化性能,并优于最先进的合成数据增强。最后,我们引入了一个感知图像增强框架,该框架能够实现分辨率增强。

英文摘要:

The inference stage of diffusion models can be seen as running a reverse-time diffusion stochastic differential equation, where samples from a Gaussian latent distribution are transformed into samples from a target distribution that usually reside on a low-dimensional manifold, e.g., an image manifold. The intermediate values between the initial latent space and the image manifold can be interpreted as noisy images, with the amount of noise determined by the forward diffusion process noise schedule. We utilize this interpretation to present Boomerang, an approach for local sampling of image manifolds. As implied by its name, Boomerang local sampling involves adding noise to an input image, moving it closer to the latent space, and then mapping it back to the image manifold through a partial reverse diffusion process. Thus, Boomerang generates images on the manifold that are ``similar,'' but nonidentical, to the original input image. We can control the proximity of the generated images to the original by adjusting the amount of noise added. Furthermore, due to the stochastic nature of the reverse diffusion process in Boomerang, the generated images display a certain degree of stochasticity, allowing us to obtain local samples from the manifold without encountering any duplicates. Boomerang offers the flexibility to work seamlessly with any pretrained diffusion model, such as Stable Diffusion, without necessitating any adjustments to the reverse diffusion process. We present three applications for Boomerang. First, we provide a framework for constructing privacy-preserving datasets having controllable degrees of anonymity. Second, we show that using Boomerang for data augmentation increases generalization performance and outperforms state-of-the-art synthetic data augmentation. Lastly, we introduce a perceptual image enhancement framework, which enables resolution enhancement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑