Let $(X, S, Y) \in \mathbb{R}^p \times \{1, 2\} \times \mathbb{R}$ be a triplet following some joint distribution $\mathbb{P}$ with feature vector $X$, sensitive attribute $S$ , and target variable $Y$. The Bayes optimal prediction $f^*$ which does not produce Disparate Treatment is defined as $f^*(x) = \mathbb{E}[Y | X = x]$. We provide a non-trivial example of a prediction $x \to f(x)$ which satisfies two common group-fairness notions: Demographic Parity \begin{align} (f(X) | S = 1) &\stackrel{d}{=} (f(X) | S = 2) \end{align} and Equal Group-Wise Risks \begin{align}
\mathbb{E}[(f^*(X) - f(X))^2 | S = 1] = \mathbb{E}[(f^*(X) - f(X))^2 | S = 2]. \end{align} To the best of our knowledge this is the first explicit construction of a non-constant predictor satisfying the above. We discuss several implications of this result on better understanding of mathematical notions of algorithmic fairness.
University of Arkansas at Fayetteville（阿肯色大学费耶特维尔分校）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
Both fair machine learning and adversarial learning have been extensively studied. However, attacking fair machine learning models has received less attention. In this paper, we present a framework that seeks to effectively generate poisoning samples to attack both model accuracy and algorithmic fairness. Our attacking framework can target fair machine learning models trained with a variety of group based fairness notions such as demographic parity and equalized odds. We develop three online attacks, adversarial sampling , adversarial labeling, and adversarial feature modification. All three a…
LinkedIn Corporation（领英公司）；Massachusetts Institute of Technology（麻省理工学院）
The impossibility theorem of fairness is a foundational result in the algorithmic fairness literature. It states that outside of special cases, one cannot exactly and simultaneously satisfy all three common and intuitive definitions of fairness - demographic parity, equalized odds, and predictive rate parity. This result has driven most works to focus on solutions for one or two of the metrics. Rather than follow suit, in this paper we present a framework that pushes the limits of the impossibility theorem in order to satisfy all three metrics to the best extent possible. We develop an integer…
We mathematically compare four competing definitions of group-level nondiscrimination: demographic parity, equalized odds, predictive parity, and calibration. Using the theoretical framework of Friedler et al., we study the properties of each definition under various worldviews, which are assumptions about how, if at all, the observed data is biased. We argue that different worldviews call for different definitions of fairness, and we specify the worldviews that, when combined with the desire to avoid a criterion for discrimination that we call disparity amplification, motivate demographic par…
On the Fairness ROAD: Robust Optimization for Adversarial Debiasing
AXA Group Operations（安盛集团运营部）；Stanford University（斯坦福大学）；TRAIL, Sorbonne Université（索邦大学TRAIL）；LERIA, Université d’Angers（昂热大学LERIA）；Polish Academy of Science（波兰科学院）
Effective machine learning models can automatically learn useful information from a large quantity of data and provide decisions in a high accuracy. These models may, however, lead to unfair predictions in certain sense among the population groups of interest, where the grouping is based on such sensitive attributes as race and gender. Various fairness definitions, such as demographic parity and equalized odds, were proposed in prior art to ensure that decisions guided by the machine learning models are equitable. Unfortunately, the "fair" model trained with these fairness definitions is thres…
Are demographically invariant models and representations in medical imaging fair?
Technical University of Denmark（丹麦技术大学）；Pioneer Centre for AI（人工智能先锋中心）；Research Institute for Signals, Systems and Computational Intelligence (CONICET)（信号、系统与计算智能研究所（CONICET））；Universidad Nacional del Litoral（国立 littoral 大学）；University of Copenhagen（哥本哈根大学）；Neurobiology Research Unit, Rigshospitalet（哥本哈根大学医院神经生物学研究单元）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Automated decision making systems are increasingly being used in real-world applications. In these systems for the most part, the decision rules are derived by minimizing the training error on the available historical data. Therefore, if there is a bias related to a sensitive attribute such as gender, race, religion, etc. in the data, say, due to cultural/historical discriminatory practices against a certain demographic, the system could continue discrimination in decisions by including the said bias in its decision rule. We present an information theoretic framework for designing fair predict…
Ethical and Explainable AI in Reusable MLOps Pipelines
Washington University of Science and Technology（科学与技术华盛顿大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Karlstad University（卡尔斯塔德大学）；American University of Bahrain（巴林美国大学）
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
In classification models fairness can be ensured by solving a constrained optimization problem. We focus on fairness constraints like Disparate Impact, Demographic Parity, and Equalized Odds, which are non-decomposable and non-convex. Researchers define convex surrogates of the constraints and then apply convex optimization frameworks to obtain fair classifiers. Surrogates serve only as an upper bound to the actual constraints, and convexifying fairness constraints might be challenging. We propose a neural network-based framework, \emph{FNNC}, to achieve fairness while maintaining high accurac…
Achieving the Bayes optimal binary classification rule subject to group fairness constraints is known to be reducible, in some cases, to learning a group-wise thresholding rule over the Bayes regressor. In this paper, we extend this result by proving that, in a broader setting, the Bayes optimal fair learning rule remains a group-wise thresholding rule over the Bayes regressor but with a (possible) randomization at the thresholds. This provides a stronger justification to the post-processing approach in fair classification, in which (1) a predictor is learned first, after which (2) its output …
Shenzhen Research Institute of Big Data（深圳大数据研究院）；University of Pennsylvania（宾夕法尼亚大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Increasing concerns about disparate effects of AI have motivated a great deal of work on fair machine learning. Existing works mainly focus on independence- and separation-based measures (e.g., demographic parity, equality of opportunity, equalized odds), while sufficiency-based measures such as predictive parity are much less studied. This paper considers predictive parity, which requires equalizing the probability of success given a positive prediction among different protected groups. We prove that, if the overall performances of different groups vary only moderately, all fair Bayes-optimal…
Tsinghua University（清华大学）；Zhejiang University（浙江大学）；Squirrel AI Learning（松鼠AI智适应教育）
Nowadays fairness issues have raised great concerns in decision-making systems. Various fairness notions have been proposed to measure the degree to which an algorithm is unfair. In practice, there frequently exist a certain set of variables we term as fair variables, which are pre-decision covariates such as users' choices. The effects of fair variables are irrelevant in assessing the fairness of the decision support algorithm. We thus define conditional fairness as a more sound fairness metric by conditioning on the fairness variables. Given different prior knowledge of fair variables, we de…
Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness
Department of Electrical and Computer Engineering, Queen’s University（皇后大学电气与计算机工程系）；Department of Computer Science and Technology, University of Cambridge（剑桥大学计算机科学与技术系）
