arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10401cs.LGcs.CV
收藏

ManiFPT:定义与分析生成模型的指纹

ManiFPT: Defining and Analyzing Fingerprints of Generative Models

USC Information Sciences Institute(南加州大学信息科学研究所);USC Ming Hsieh Department of Electrical and Computer Engineering(南加州大学明赫氏电气与计算机工程系)

Hae Jin Song, Mahyar Khayatkhoei, Wael AbdAlmageed

AI总结:

本文形式化生成模型伪影与指纹定义并提出计算算法,用于区分多种生成模型和模型归因,且指纹结构可预测设计选择影响。

摘要:

近期研究表明,生成模型会在生成样本中留下其底层生成过程的痕迹,这些痕迹被广义地称为生成模型的指纹,并且已有研究考察了它们在检测合成图像与真实图像方面的效用。然而,这些指纹在多大程度上能够区分不同类型的合成图像,并帮助识别底层生成过程,仍未得到充分探索。特别是,据我们所知,指纹本身的定义仍不明确。为此,在本工作中,我们形式化了生成模型中伪影与指纹的定义,提出了一种在实践中计算它们的算法,并最终研究了该算法在区分大量不同生成模型时的有效性。我们发现,与现有方法相比,使用我们提出的定义能够显著提升从样本中识别底层生成过程(即模型归因)任务的性能。此外,我们研究了指纹的结构,并观察到该结构对不同设计选择在生成过程上的影响具有很强的预测能力。

英文摘要:

Recent works have shown that generative models leave traces of their underlying generative process on the generated samples, broadly referred to as fingerprints of a generative model, and have studied their utility in detecting synthetic images from real ones. However, the extend to which these fingerprints can distinguish between various types of synthetic image and help identify the underlying generative process remain under-explored. In particular, the very definition of a fingerprint remains unclear, to our knowledge. To that end, in this work, we formalize the definition of artifact and fingerprint in generative models, propose an algorithm for computing them in practice, and finally study its effectiveness in distinguishing a large array of different generative models. We find that using our proposed definition can significantly improve the performance on the task of identifying the underlying generative process from samples (model attribution) compared to existing methods. Additionally, we study the structure of the fingerprints, and observe that it is very predictive of the effect of different design choices on the generative process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑