SAFREE：用于安全文本到图像和视频生成的免训练自适应防护机制
SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation
AI总结：
本文提出免训练的 SAFREE 防护框架，通过有毒概念子空间引导、自验证过滤和自适应重注意机制，在不改变扩散模型权重的情况下提升 T2I/T2V 生成安全性并保持图像质量。
摘要：
扩散模型的最新进展显著增强了其生成高质量图像和视频的能力，但也增加了产生不安全内容的风险。现有的基于遗忘/编辑的安全生成方法会从模型中移除有害概念，但面临若干挑战：（1）它们无法在不进行训练的情况下即时移除有害概念。（2）其安全生成能力依赖于收集的训练数据。（3）它们会改变模型权重，存在导致与有毒概念无关内容质量下降的风险。为解决这些问题，我们提出了 SAFREE，这是一种新颖的、免训练的安全 T2I 和 T2V 方法，不会改变模型权重。具体而言，我们在文本嵌入空间中检测与一组有毒概念对应的子空间，并引导提示嵌入远离该子空间，从而在过滤有害内容的同时保留预期语义。为平衡过滤毒性与保留安全概念之间的权衡，SAFREE 纳入了一种新颖的自验证过滤机制，该机制在应用过滤后的嵌入时动态调整去噪步骤。此外，我们在扩散潜空间中加入自适应重注意机制，以在像素层面选择性削弱与有毒概念相关特征的影响。最终，SAFREE 确保连贯的安全检查，保留输出的保真度、质量和安全性。与免训练基线相比，SAFREE 在抑制 T2I 生成中的不安全内容方面取得了 SOTA 性能，并在保持高质量图像的同时有效过滤目标概念。它还展现出与基于训练的方法相比具有竞争力的结果。我们将 SAFREE 扩展到各种 T2I 骨干网络和 T2V 任务，展示了其灵活性和泛化能力。SAFREE 为确保安全视觉生成提供了稳健且适应性强的防护措施。
英文摘要：
Recent advances in diffusion models have significantly enhanced their ability to generate high-quality images and videos, but they have also increased the risk of producing unsafe content. Existing unlearning/editing-based methods for safe generation remove harmful concepts from models but face several challenges: (1) They cannot instantly remove harmful concepts without training. (2) Their safe generation capabilities depend on collected training data. (3) They alter model weights, risking degradation in quality for content unrelated to toxic concepts. To address these, we propose SAFREE, a novel, training-free approach for safe T2I and T2V, that does not alter the model's weights. Specifically, we detect a subspace corresponding to a set of toxic concepts in the text embedding space and steer prompt embeddings away from this subspace, thereby filtering out harmful content while preserving intended semantics. To balance the trade-off between filtering toxicity and preserving safe concepts, SAFREE incorporates a novel self-validating filtering mechanism that dynamically adjusts the denoising steps when applying the filtered embeddings. Additionally, we incorporate adaptive re-attention mechanisms within the diffusion latent space to selectively diminish the influence of features related to toxic concepts at the pixel level. In the end, SAFREE ensures coherent safety checking, preserving the fidelity, quality, and safety of the output. SAFREE achieves SOTA performance in suppressing unsafe content in T2I generation compared to training-free baselines and effectively filters targeted concepts while maintaining high-quality images. It also shows competitive results against training-based methods. We extend SAFREE to various T2I backbones and T2V tasks, showcasing its flexibility and generalization. SAFREE provides a robust and adaptable safeguard for ensuring safe visual generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
