MemBench:用于扩散模型的记忆图像触发提示数据集
MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Models
AI总结:
本文提出首个图像记忆缓解评估基准MemBench,包含多种扩散模型的触发提示,并设计同时评估触发提示与通用提示的指标,验证现有缓解方法仍不足以实际应用。
摘要:
扩散模型在文本到图像生成任务中取得了显著成功,推动了众多商业模型的发展。然而,近期研究指出,扩散模型在特定提示触发时,往往会生成训练数据中的复制图像,可能引发从版权到隐私等一系列社会问题。为规避这种记忆现象,已有研究致力于开发针对扩散模型的记忆缓解方法。然而,缺乏基准测试阻碍了对这些方法真实有效性的评估。在本工作中,我们提出了MemBench,这是首个用于评估图像记忆缓解方法的基准。我们的基准包含大量在各种文本到图像扩散模型中记忆图像触发提示。此外,与先前仅基于触发提示评估缓解性能的工作不同,我们提出了同时在触发提示和通用提示上进行评估的指标,以便观察缓解方法是否在解决记忆问题的同时保持通用提示下的性能。考虑到先前工作所忽视的实际应用场景,这是一项重要进展。通过在MemBench上的评估,我们验证了现有图像记忆缓解方法在扩散模型上的应用性能仍然不足。代码和数据集可在 https://github.com/chunsanHong/MemBench_code 获取。
英文摘要:
Diffusion models have achieved remarkable success in Text-to-Image generation tasks, leading to the development of many commercial models. However, recent studies have reported that diffusion models often generate replicated images in train data when triggered by specific prompts, potentially raising social issues ranging from copyright to privacy concerns. To sidestep the memorization, there have been recent studies for developing memorization mitigation methods for diffusion models. Nevertheless, the lack of benchmarks impedes the assessment of the true effectiveness of these methods. In this work, we present MemBench, the first benchmark for evaluating image memorization mitigation methods. Our benchmark includes a large number of memorized image trigger prompts in various Text-to-Image diffusion models. Furthermore, in contrast to the prior work evaluating mitigation performance only on trigger prompts, we present metrics evaluating on both trigger prompts and general prompts, so that we can see whether mitigation methods address the memorization issue while maintaining performance for general prompts. This is an important development considering the practical applications which previous works have overlooked. Through evaluation on MemBench, we verify that the performance of existing image memorization mitigation methods is still insufficient for application to diffusion models. The code and datasets are available at https://github.com/chunsanHong/MemBench\_code.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。