arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.02097cs.CV
收藏

在扩散模型中通过初始化保留图像属性

Preserving Image Properties Through Initializations in Diffusion Models

Jeffrey Zhang, Shao-Yu Chang, Kedan Li, David Forsyth

AI总结:

研究Stable Diffusion在零售图像生成中背景等属性不一致问题,提出通过近似噪声分布初始化推理并训练网络,显著提升生成质量与可控性。

摘要:

零售摄影对图像有特定要求。例如,图像可能需要统一的背景颜色、一致的模特姿势、居中的产品以及一致的照明。与这些标准的微小偏差会影响网站的美观度,使图像无法使用。我们表明,Stable Diffusion 方法在当前应用中并未遵守这些要求。通常的做法是用非常嘈杂的图像训练去噪器,并以纯噪声样本开始推理,这会导致推理过程中生成不一致的图像。这种不一致性是因为很容易区分训练分布和推理分布的样本。结果,用具有统一背景的居中零售产品图像训练的网络会生成背景杂乱的图像。该问题可以通过使用近似嘈杂图像的样本初始化推理来轻松修复。然而,在使用这种近似时,推理时的文本和嘈杂图像的联合分布仍与训练时略有不同。这种差异通过用近似嘈杂图像分布的样本训练网络来纠正。在真实应用数据上的大量实验表明,采用这些程序在性能上取得了显著的定性和定量改进。最后,我们的方法能与其他基于控制的方法良好交互,进一步增强基于扩散方法的可控性。

英文摘要:

Retail photography imposes specific requirements on images. For instance, images may need uniform background colors, consistent model poses, centered products, and consistent lighting. Minor deviations from these standards impact a site's aesthetic appeal, making the images unsuitable for use. We show that Stable Diffusion methods, as currently applied, do not respect these requirements. The usual practice of training the denoiser with a very noisy image and starting inference with a sample of pure noise leads to inconsistent generated images during inference. This inconsistency occurs because it is easy to tell the difference between samples of the training and inference distributions. As a result, a network trained with centered retail product images with uniform backgrounds generates images with erratic backgrounds. The problem is easily fixed by initializing inference with samples from an approximation of noisy images. However, in using such an approximation, the joint distribution of text and noisy image at inference time still slightly differs from that at training time. This discrepancy is corrected by training the network with samples from the approximate noisy image distribution. Extensive experiments on real application data show significant qualitative and quantitative improvements in performance from adopting these procedures. Finally, our procedure can interact well with other control-based methods to further enhance the controllability of diffusion-based methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑