arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.03397cs.AIcs.CV
收藏

使用概念图在文本到图像扩散模型中有效减轻偏差

Efficient bias mitigation in T2I diffusion models using Concept Graphs

Department of Computing Imperial College London(伦敦帝国理工学院计算系)

Mansi, Avinash Kori, Francesco Leofante

AI总结:

研究文本到图像扩散模型偏差问题,提出基于概念图对齐的CO-ALIGN方法,在模型内部概念本体上操作,能有效减轻偏差并保留生成完整性,性能优于现有技术。

摘要:

文本到图像扩散模型常传播训练数据中的有害偏差。现有减轻偏差技术有局限。本文引入CO-ALIGN,基于概念图对齐的新方法,在模型内部概念本体上运行。通过对齐文本编码器和去噪器中的概念,大幅减少偏差并保持生成完整性,在多范式中有效,性能优于现有技术,还利于其他下游任务。

英文摘要:

Text-to-Image diffusion models often propagate harmful bias inherited from the training data. Existing bias mitigation techniques typically intervene only at the text encoder or provide inference-time guidance, often leading to generations that collapse into semantically incoherent outputs. To address these limitations, we introduce CO-ALIGN (Concept Ontology Alignment), a novel bias mitigation approach based on concept-graph alignment that operates on the model's internal concept ontology. By aligning concepts within the text encoder and denoiser, CO-ALIGN achieves substantial bias reduction while preserving generative integrity. We demonstrate the effectiveness of concept-graph alignment across three paradigms: text-encoders, denoisers and joint text-denoiser ontology alignment. CO-ALIGN outperforms the state of the art, improving fairness by $30\%$, $ΔFID=11.4$ in image quality, $2.8\%$ in image fidelity, all while reducing semantically incoherent outputs by $88\%$. Beyond bias mitigation, we show that CO-ALIGN benefits other downstream tasks as well. In particular, our experiments demonstrate that better-aligned internal ontologies enhance concept unlearning robustness across multiple unlearning techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑