立场：生成式模型中的公平性失效是一个评估问题
Position: Fairness Failure in Generative Models is an Evaluation Problem
浏览论文内容
中文总结 AI 辅助
本文指出生成式模型的公平性失效源于评估问题，提出Fairness Cards工具明确评估选择以提升公平性评估的可复现性与可比性，推动评估标准的范式转变。
中文摘要 AI 辅助
尽管过去十年生成式模型取得了突破性进展，但人们对其缺乏公平性、加剧社会不平等并伤害边缘群体的担忧仍未得到充分解决，且难以落实行动。本立场论文指出，生成式模型的公平性失效虽由多种因素驱动，但最终源于一个评估问题：不同论文的公平性发现极少具有可比性，也难以用于部署决策。本文诊断了当前实践中反复出现的经验性和概念性失效模式，推动从临时偏见检查转向针对生成式模型的标准化评估。我们提出Fairness Cards作为一种最小报告工具，可明确评估选择（提示系列、反事实协议、指标及弃权（不执行）处理方式），实现可复现性、可比性和问责制。最后，我们提出额外建议，推动评估标准的范式转变。本项目页面可通过此https URL获取。
英文摘要
Despite groundbreaking advancements in generative models during the last decade, concerns about their lack of fairness, reinforcing societal inequalities and harming marginalized groups, remain under-addressed and difficult to act upon. This position paper argues that fairness failures in generative models, albeit driven by multiple factors, are ultimately stemming from an evaluation problem: fairness findings are rarely comparable across papers or actionable for deployment decisions. This paper diagnoses recurring empirical and conceptual failure modes in current practice and motivates a shift from ad-hoc bias checks to standardized, generative-specific evaluation. We propose Fairness Cards as a minimal reporting artifact that makes evaluation choices explicit (prompt families, counterfactual protocols, metrics, and refusal handling) enabling reproducibility, comparability, and accountability. We conclude with additional recommendations towards a paradigm shift in evaluation standards. Our project page can be found at https://mariiavladimirova.github.io/fairness-cards .
发表机构
- Criteo AI Lab（Criteo人工智能实验室）
- FairPlay joint team（FairPlay联合团队）
机构由 AI 辅助整理，请以论文原文为准。
