arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10882cs.CVcs.CL
收藏

面向安全文本生成图像的通用提示优化器

Universal Prompt Optimizer for Safe Text-to-Image Generation

The Pennsylvania State University(宾夕法尼亚州立大学);Tianjin University(天津大学);University of Chinese Academy of Sciences(中国科学院大学)

Zongyu Wu, Hongcheng Gao, Yueze Wang, Xiang Zhang, Suhang Wang

AI总结:

针对文本生成图像模型易生成不安全内容的问题,提出黑盒场景下的通用提示优化器POSI,通过构建有毒-干净提示对数据集、设计新型奖励函数并结合近端策略优化训练,有效降低不当图像生成概率且不影响文本对齐度。

摘要:

文本生成图像(Text-to-Image, T2I)模型在根据文本提示生成图像方面已展现出优异性能。然而,这类模型易受不安全输入影响,生成色情、骚扰、非法活动类等不安全内容。现有基于图像检测器、模型微调、嵌入阻断的研究在实际应用中并不实用。为此,我们提出了首个面向黑盒场景下安全T2I生成的通用提示优化器POSI。我们首先通过GPT-3.5 Turbo构建了包含有毒-干净提示对的数据集。为使优化器具备在保留语义信息的前提下将有毒提示转换为干净提示的能力,我们设计了一种衡量生成图像毒性与文本对齐度的新型奖励函数,并通过近端策略优化(Proximal Policy Optimization)训练优化器。实验表明,我们的方法能有效降低各类T2I模型生成不当图像的概率,且不会对文本对齐度造成显著影响,还可灵活与其他方法结合以实现更优性能。我们的代码可在https://github.com/wu-zongyu/POSI 获取。

英文摘要:

Text-to-Image (T2I) models have shown great performance in generating images based on textual prompts. However, these models are vulnerable to unsafe input to generate unsafe content like sexual, harassment and illegal-activity images. Existing studies based on image checker, model fine-tuning and embedding blocking are impractical in real-world applications. Hence, we propose the first universal prompt optimizer for safe T2I (POSI) generation in black-box scenario. We first construct a dataset consisting of toxic-clean prompt pairs by GPT-3.5 Turbo. To guide the optimizer to have the ability of converting toxic prompt to clean prompt while preserving semantic information, we design a novel reward function measuring toxicity and text alignment of generated images and train the optimizer through Proximal Policy Optimization. Experiments show that our approach can effectively reduce the likelihood of various T2I models in generating inappropriate images, with no significant impact on text alignment. It is also flexible to be combined with methods to achieve better performance. Our code is available at https://github.com/wu-zongyu/POSI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑