arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.00616cs.AI
收藏

SPOT:基于总变分的选性提示投影用于仅推理的文本到图像生成

SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generation

Seoul National University(首尔国立大学)

Minhyuk Lee, Hyekyung Yoon, Myungjoo Kang

AI总结:

本文提出SPOT框架,通过总变分约束生成器参考分布,实现对文本到图像生成中不安全内容的抑制,同时保持良性提示的行为,实验显示其在多个数据集上显著降低不适当评分。

摘要:

文本到图像(T2I)扩散模型能够生成高质量的开放生成内容,但实际应用中需要在抑制不安全生成的同时保持对良性提示的行为。本文研究了在冻结生成器的情况下,通过其提示条件分布作为保持参考的张力。由于T2I安全性通常通过生成图像上的有界风险分数来评估,总变分(TV)限制了从该参考期望风险的变化幅度。我们称之为固定参考约束为安全提示对齐权衡(SPAT):减少期望不安全性需要提示条件分布的偏离。为了使这种偏离具有选择性和可调性,我们定义tau安全集为参考风险不超过tau的提示,并将干预作为在该集合中附近提示的投影。我们提出选性提示投影(SPOT),一种推理时间框架,该框架在不重新训练生成器或学习特定类别的重写器的情况下近似这种投影。SPOT使用LLM对候选重写进行排序,并使用一个保护性的视觉语言模型(VLM)在相同tau下接受生成的图像。在四个数据集和三个扩散基础架构上,SPOT在强安全对齐基线上实现了不适当(IP)评分的相对减少,从14.2%到44.4%,同时保持良性提示行为接近固定参考。

英文摘要:

Text-to-Image (T2I) diffusion models enable high quality open ended synthesis, but practical use requires suppressing unsafe generations while preserving behavior on benign prompts. We study this tension relative to the frozen generator, using its prompt conditioned distribution as the preservation reference. Since T2I safety is commonly evaluated by bounded risk scores on generated images, total variation (TV) bounds how much expected risk can change from this reference. We call this fixed reference constraint the Safety-Prompt Alignment Tradeoff (SPAT): reducing expected unsafety requires prompt conditioned distributional deviation. To make this deviation selective and adjustable, we define the tau safe set as prompts whose reference risk is at most tau, and cast intervention as projection toward nearby prompts in this set. We propose Selective Prompt prOjecTion (SPOT), an inference time framework that approximates this projection without retraining the generator or learning a category specific rewriter. SPOT uses an LLM to rank candidate rewrites and a safeguard VLM to accept generated images under the same tau. Across four datasets and three diffusion backbones, SPOT achieves relative inappropriate (IP) score reductions from 14.2% to 44.4% over strong safety alignment baselines while keeping benign prompt behavior close to the fixed reference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑