发表机构
Criteo AI Lab; ESSEC Business School; CentraleSupélec, Université Paris-Saclay; FairPlay joint team（Criteo AI实验室; ESSEC商学院; 巴黎萨克雷大学中央理工-高等电力学院; FairPlay联合团队）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种通用适配器方法，联合优化扩散模型的公平性与多样性，无需敏感属性标注，通过文本提示方向和语义不一致性分数实现，实验证明在同等公平性下提升质量与多样性。
AI 中文摘要
尽管扩散模型能够生成高质量的图像，但它们也会再现并放大训练数据中的人口统计失衡。在训练后针对某些敏感属性对其生成过程进行去偏，通常依赖于分类器引导或显式的文本额外条件化，但这降低了方法的适用性和输出多样性。相反，仅促进多样性的方法并不能确保公平的属性表示。在本文中，我们提出了一种联合处理公平性和多样性的方法，该方法普遍适用于任何扩散模型和任何敏感属性。为此，一个适配器将冻结的扩散模型连接到预训练的视觉-语言嵌入空间，从而在没有敏感属性标注的情况下实现公平性和多样性引导。对于公平性，成对的文本提示定义属性方向，引导批次组成朝向特定比例。对于多样性，我们引入了一个分数，用于衡量从该表示中获得的语义估计之间的不一致性。该公式支持无条件扩散模型和文本条件扩散模型，同时不需要关于敏感属性的先验知识或数据。实验证实，我们的方法在相当的公平性水平下提高了质量和多样性分数。
英文摘要
Although diffusion models produce high-quality images, they also reproduce and amplify demographic imbalances in their training data. Debiasing their generation process post-training w.r.t. some sensitive attribute usually relies on classifier guidance or explicit text extra-conditioning, but this reduces methods' applicability and output diversity. Conversely, methods promoting diversity alone do not ensure fair attribute representation. In this paper, we propose a method tackling fairness and diversity jointly that is generally applicable to any diffusion model and any sensitive attribute. To this end, an adapter connects the frozen diffusion model to a pretrained vision-language embedding space, enabling fairness and diversity guidance without sensitive-attribute annotations. For fairness, pairs of text prompts define attribute directions which guide batch composition towards specific proportions. For diversity, we introduce a score measuring disagreement between the semantic estimates derived from this representation. The formulation supports unconditional and text-conditional diffusion models, while requiring no prior knowledge or data of sensitive attribute. Experiments confirm that our method improves quality and diversity scores at comparable fairness levels.
