UnsafeBench：在真实世界与AI生成图像上对图像安全分类器的基准测试
UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images
AI总结：
针对图像安全分类器在真实与AI生成图像上性能未知的问题，提出UnsafeBench基准测试框架，评估发现现有分类器不够全面有效且存在分布偏移，并构建PerspectiveVision工具提升其有效性与鲁棒性。
摘要：
随着文本到图像模型的出现及其被滥用的担忧，开发者们越来越依赖图像安全分类器来审核其生成的的不安全图像。然而，当前图像安全分类器在真实世界和AI生成图像上的性能仍然未知。在这项工作中，我们提出了UnsafeBench，一个评估图像安全分类器有效性和鲁棒性的基准测试框架，特别关注AI生成图像对其性能的影响。首先，我们整理了一个包含10K张真实世界和AI生成图像的大型数据集，这些图像根据11类不安全图像类别（如色情、暴力、仇恨等）被标注为安全或不安全。然后，我们评估了五个流行的图像安全分类器以及三个由通用视觉语言模型驱动的分类器的有效性和鲁棒性。我们的评估表明，现有的图像安全分类器在缓解不安全图像的多方面问题上不够全面和有效。此外，真实世界图像与AI生成图像在图像质量、风格和布局上存在分布偏移，导致有效性和鲁棒性下降。基于这些发现，我们构建了一个名为PerspectiveVision的全面图像审核工具，它提高了现有分类器的有效性和鲁棒性，尤其是在AI生成图像上。UnsafeBench和PerspectiveVision可以帮助研究社区更好地理解生成式AI时代图像安全分类的格局。
英文摘要：
With the advent of text-to-image models and concerns about their misuse, developers are increasingly relying on image safety classifiers to moderate their generated unsafe images. Yet, the performance of current image safety classifiers remains unknown for both real-world and AI-generated images. In this work, we propose UnsafeBench, a benchmarking framework that evaluates the effectiveness and robustness of image safety classifiers, with a particular focus on the impact of AI-generated images on their performance. First, we curate a large dataset of 10K real-world and AI-generated images that are annotated as safe or unsafe based on a set of 11 unsafe categories of images (sexual, violent, hateful, etc.). Then, we evaluate the effectiveness and robustness of five popular image safety classifiers, as well as three classifiers that are powered by general-purpose visual language models. Our assessment indicates that existing image safety classifiers are not comprehensive and effective enough to mitigate the multifaceted problem of unsafe images. Also, there exists a distribution shift between real-world and AI-generated images in image qualities, styles, and layouts, leading to degraded effectiveness and robustness. Motivated by these findings, we build a comprehensive image moderation tool called PerspectiveVision, which improves the effectiveness and robustness of existing classifiers, especially on AI-generated images. UnsafeBench and PerspectiveVision can aid the research community in better understanding the landscape of image safety classification in the era of generative AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
