arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.14756cs.CV
收藏

量化并提升扩散模型生成图像的真实感

On quantifying and improving realism of images generated with diffusion

The University of Western Australia(西澳大利亚大学)

Yunzhuo Chen, Naveed Akhtar, Nur Al Hasan Haldar, Ajmal Mian

AI总结:

本文提出非学习的单图真实感指标 IRS,可跨模型与数据检测扩散及生成模型伪造图像,并用于优化 SDM 损失以提升生成质量,同时发布 Gen-100 数据集与代码。

摘要:

扩散模型的最新进展使生成视觉内容的质量实现了巨大飞跃。然而,量化这些内容的真实感仍具挑战性。现有评估指标(如 Inception Score 和 Fréchet inception distance)由于生成图像的多样性,在对扩散模型进行基准测试时存在不足。此外,它们并非为量化单张图像的真实感而设计。这限制了其在取证图像分析中的应用,而在生成模型兴起的时代,此类分析正变得日益重要。为解决该问题,我们首先提出一种称为 Image Realism Score(IRS,图像真实感分数)的指标,该指标由给定图像的五项统计度量计算得出。这种基于非学习的指标不仅能高效量化生成图像的真实感,还可直接用作将给定图像分类为真实或伪造的度量。我们通过实验成功检测由 Stable Diffusion Model(SDM)、Dalle2、Midjourney 和 BigGAN 生成的伪造图像,确立了所提出 IRS 的模型无关和数据无关特性。我们进一步利用该指标的这一属性,最小化 SDM 的 IRS 增强生成损失,并证明通过我们的修改,SDM 生成内容获得了便捷且显著的质量提升。我们的工作还形成了 Gen-100 数据集,该数据集为 100 个类别提供由四个高质量模型生成的 1,000 个样本。我们将公开该数据集和代码。

英文摘要:

Recent advances in diffusion models have led to a quantum leap in the quality of generative visual content. However, quantification of realism of the content is still challenging. Existing evaluation metrics, such as Inception Score and Fréchet inception distance, fall short on benchmarking diffusion models due to the versatility of the generated images. Moreover, they are not designed to quantify realism of an individual image. This restricts their application in forensic image analysis, which is becoming increasingly important in the emerging era of generative models. To address that, we first propose a metric, called Image Realism Score (IRS), computed from five statistical measures of a given image. This non-learning based metric not only efficiently quantifies realism of the generated images, it is readily usable as a measure to classify a given image as real or fake. We experimentally establish the model- and data-agnostic nature of the proposed IRS by successfully detecting fake images generated by Stable Diffusion Model (SDM), Dalle2, Midjourney and BigGAN. We further leverage this attribute of our metric to minimize an IRS-augmented generative loss of SDM, and demonstrate a convenient yet considerable quality improvement of the SDM-generated content with our modification. Our efforts have also led to Gen-100 dataset, which provides 1,000 samples for 100 classes generated by four high-quality models. We will release the dataset and code.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑