Text-to-Image (T2I) models have shown great performance in generating images based on textual prompts. However, these models are vulnerable to unsafe input to generate unsafe content like sexual, harassment and illegal-activity images. Existing studies based on image checker, model fine-tuning and embedding blocking are impractical in real-world applications. Hence, we propose the first universal prompt optimizer for safe T2I (POSI) generation in black-box scenario. We first construct a dataset consisting of toxic-clean prompt pairs by GPT-3.5 Turbo. To guide the optimizer to have the ability of converting toxic prompt to clean prompt while preserving semantic information, we design a novel reward function measuring toxicity and text alignment of generated images and train the optimizer through Proximal Policy Optimization. Experiments show that our approach can effectively reduce the likelihood of various T2I models in generating inappropriate images, with no significant impact on text alignment. It is also flexible to be combined with methods to achieve better performance. Our code is available at https://github.com/wu-zongyu/POSI.
PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generation
Beihang University(北航);Beijing University of Posts and Telecommunications(北京邮电大学);Taishan University(泰山大学);Nanyang Technological University(南洋理工大学);Henan University of Science and Technology(河南科技大学)
SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generation
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea(电气与计算机工程系,首尔国立大学,首尔,韩国);IPAI / ASRI / INMC, Seoul National University, Seoul, South Korea(IPAI/ASRI/INMC,首尔国立大学,首尔,韩国)
PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models
Department of Computer Science, University of Maryland(计算机科学系,马里兰大学);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算机科学与数据科学学院);Kahlert School of Computing, The University of Utah(犹他大学Kahlert计算学院);Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院)
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models
MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Sony(索尼);Weizmann Institute of Science(魏茨曼科学研究院);JKU(约翰纳斯堡大学);Tübingen AI Center(图宾根人工智能中心);UVA(乌得勒支大学);UNSW(新南威尔士大学);TU Graz(格拉茨技术大学);MIT-IBM(麻省理工学院-IBM)
The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);School of Computing and Information Systems, Singapore Management University(新加坡管理学院计算与信息系统学院)
PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
Posts and Telecommunications Institute of Technology(邮电技术研究所);Hanoi Architectural University(河内建筑大学)