arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2505.08804cs.CRcs.LG
收藏

TokenProber:通过细粒度词语影响分析越狱文本到图像模型

TokenProber: Jailbreaking Text-to-image Models via Fine-grained Word Impact Analysis

School of Cyber Science and Engineering, Xi’an Jiaotong University(网络安全科学与工程学院,西安交通大学);Singapore Management University(新加坡管理学院);Nanyang Technological University(南洋理工大学)

Longtian Wang, Xiaofei Xie, Tianlin Li, Yuhan Zhi, Chao Shen

AI总结:

本文提出TokenProber,通过细粒度分析提示词语对T2I模型与安全检查器的敏感度差异并进行变异,高效生成对抗性提示以揭示安全拒绝机制漏洞。

摘要:

文本到图像(T2I)模型在生成高质量图像方面已取得显著进展。然而,此类模型能够生成包含不适合工作场所(NSFW)内容的图像,例如色情、暴力、政治内容和歧视。为降低生成NSFW内容的风险,研究人员开发了拒绝机制(即安全检查器),用于检查潜在的NSFW内容。对抗性提示技术也已被开发出来,用于评估拒绝机制的鲁棒性。关键挑战仍然在于,如何以一种既保留提示敏感性质又能绕过拒绝机制的方式,对提示进行微妙修改。本文提出TokenProber,这是一种面向敏感度感知差分测试的方法,旨在通过生成对抗性提示来评估T2I模型中拒绝机制的鲁棒性。我们的方法基于一个关键观察:对抗性提示之所以经常成功,是因为利用了T2I模型与安全检查器在解释敏感内容方面的差异。因此,我们对提示中特定词语的影响进行细粒度分析,区分对NSFW内容生成必不可少的脏词,以及凸显T2I模型与安全检查器之间不同敏感度评估的差异词。通过敏感度感知变异,TokenProber生成对抗性提示,在维持NSFW内容生成与规避检测之间取得平衡。我们使用324条NSFW提示,在3个流行T2I模型上针对5个安全检查器评估TokenProber,结果表明,与现有方法相比,其在绕过安全过滤器方面具有更优有效性(例如平均提升54%以上),凸显了TokenProber发现现有拒绝机制中鲁棒性问题的能力。

英文摘要:

Text-to-image (T2I) models have significantly advanced in producing high-quality images. However, such models have the ability to generate images containing not-safe-for-work (NSFW) content, such as pornography, violence, political content, and discrimination. To mitigate the risk of generating NSFW content, refusal mechanisms, i.e., safety checkers, have been developed to check potential NSFW content. Adversarial prompting techniques have been developed to evaluate the robustness of the refusal mechanisms. The key challenge remains to subtly modify the prompt in a way that preserves its sensitive nature while bypassing the refusal mechanisms. In this paper, we introduce TokenProber, a method designed for sensitivity-aware differential testing, aimed at evaluating the robustness of the refusal mechanisms in T2I models by generating adversarial prompts. Our approach is based on the key observation that adversarial prompts often succeed by exploiting discrepancies in how T2I models and safety checkers interpret sensitive content. Thus, we conduct a fine-grained analysis of the impact of specific words within prompts, distinguishing between dirty words that are essential for NSFW content generation and discrepant words that highlight the different sensitivity assessments between T2I models and safety checkers. Through the sensitivity-aware mutation, TokenProber generates adversarial prompts, striking a balance between maintaining NSFW content generation and evading detection. Our evaluation of TokenProber against 5 safety checkers on 3 popular T2I models, using 324 NSFW prompts, demonstrates its superior effectiveness in bypassing safety filters compared to existing methods (e.g., 54%+ increase on average), highlighting TokenProber's ability to uncover robustness issues in the existing refusal mechanisms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑