无需训练的扩散模型去偏方法:基于CLIP引导的去噪优化
Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization
AI总结:
提出TES框架,通过扩散过程中优化文本嵌入来缓解人口统计偏见,无需重新训练,使用两阶段策略(早期全局对齐+迭代去噪时CLIP反馈)实现稳定可控的属性引导。
摘要:
文本到图像扩散模型实现了令人印象深刻的视觉质量,但人口统计偏见仍然是一个挑战,因为中性提示词在性别和种族上始终产生刻板印象。现有方法受限于昂贵的重新训练或推理时干预,这些干预通常会降低图像质量和语义对齐。我们提出文本嵌入引导(TES),一个无需训练的框架,通过在扩散过程中直接优化条件文本嵌入来缓解人口统计偏见。我们证明,两阶段策略——早期全局对齐,随后基于CLIP反馈的迭代去噪时细化——能够在无需修改模型参数的情况下实现稳定且可控的属性引导。在Stable Diffusion上的大量实验表明,TES在公平性方面优于现有的无需训练基线,同时保持有竞争力的图像质量。这些结果突显了推理时文本嵌入优化是扩散模型中实现公平感知生成的一种实用且可扩展的解决方案。
英文摘要:
Text-to-image diffusion models achieve impressive visual quality, yet demographic bias remains a challenge, as neutral prompts consistently produce stereotypical representations across gender and race. Existing approaches remain limited by costly retraining or by inference-time interventions that often degrade image quality and semantic alignment. We propose Text Embedding Steering (TES), a training-free framework that mitigates demographic bias by directly optimizing conditional text embeddings during the diffusion process. We show that a two-stage strategy - early-stage global alignment followed by iterative denoising-time refinement with CLIP-based feedback - enables stable and controllable attribute steering without modifying model parameters. Extensive experiments on Stable Diffusion demonstrate that TES outperforms existing training-free baselines in fairness while maintaining competitive image quality. These results highlight that inference-time text embedding optimization is a practical and scalable solution for fairness-aware generation in diffusion models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。