On the Fairness of Randomized Trials for Recommendation with Heterogeneous Demographics and Beyond
Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang
备注：
The latest version of this paper has been published in NeurIPS'20 with the name: Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback. Please refer to arXiv:2009.02623
Observed events in recommendation are consequence of the decisions made by a policy, thus they are usually selectively labeled, namely the data are Missing Not At Random (MNAR), which often causes large bias to the estimate of true outcomes risk. A general approach to correct MNAR bias is performing small Randomized Controlled Trials (RCTs), where an additional uniform policy is employed to randomly assign items to each user. In this work, we concentrate on the fairness of RCTs under both homogeneous and heterogeneous demographics, especially analyzing the bias for the least favorable group on the latter setting. Considering RCTs' limitations, we propose a novel Counterfactual Robust Risk Minimization (CRRM) framework, which is totally free of expensive RCTs, and derive its theoretical generalization error bound. At last, empirical experiments are performed on synthetic tasks and real-world data sets, substantiating our method's superiority both in fairness and generalization.
Machine learning models (e.g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e.g., non-native speakers) contribute less to the training objective and thus tend to suffer higher loss. Worse, as model accuracy affects user retention, a minority group can shrink over time. In this paper, we first show that the status quo of empirical risk minimization (ERM) amplifies representation disparity over time, which can even make initially fair models unfair. To mitigate this, we develop an approach based on distributionally…
On Demographic Group Fairness Guarantees in Deep Learning
Harvard AI and Robotics Lab at Harvard University（哈佛大学人工智能与机器人实验室）；Embodied AI and Robotics (AIR) Lab at New York University（纽约大学具身人工智能与机器人实验室）；School of Computing and Informatics, University of Louisiana at Lafayette（路易斯安那州立大学拉法叶分校计算机与信息学院）
Fairness without Demographics through Learning Graph of Gradients
Carnegie Mellon University（卡内基梅隆大学）；The Chinese University of Hong Kong（香港中文大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tsinghua University（清华大学）
Heinz College of Information Systems and Public Policy, Carnegie Mellon University（卡内基梅隆大学海因茨信息系统与公共政策学院）
We propose a general approach for training survival analysis models that minimizes a worst-case error across all subpopulations that are large enough (occurring with at least a user-specified minimum probability). This approach uses a training loss function that does not know any demographic information to treat as sensitive. Despite this, we demonstrate that our proposed approach often scores better on recently established fairness metrics (without a significant drop in prediction accuracy) compared to various baselines, including ones which directly use sensitive demographic information in t…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Google Research（谷歌研究院）
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns. However, in practice factors like privacy and regulation often preclude the collection of protected features, or their use for training or inference, severely limiting the applicability of traditional fairness research. Therefore we ask: How can we train an ML model to improve fairness when we do not even know the protected group memberships? In this work we address this problem by proposing Adversari…
Grad2Fair: A Gradient-driven Approach for Graph Fairness without Demographics
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究所）
Making fair decisions is crucial to ethically implementing machine learning algorithms in social settings. In this work, we consider the celebrated definition of counterfactual fairness [Kusner et al., NeurIPS, 2017]. We begin by showing that an algorithm which satisfies counterfactual fairness also satisfies demographic parity, a far simpler fairness constraint. Similarly, we show that all algorithms satisfying demographic parity can be trivially modified to satisfy counterfactual fairness. Together, our results indicate that counterfactual fairness is basically equivalent to demographic pari…
Towards Harmless Rawlsian Fairness Regardless of Demographic Prior
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Institute of High-Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；Centre for Frontier AI Research, Agency for Science, Technology and Research（新加坡科技研究局前沿人工智能研究中心）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Classifiers that achieve demographic balance by explicitly using protected attributes such as race or gender are often politically or culturally controversial due to their lack of individual fairness, i.e. individuals with similar qualifications will receive different outcomes. Individually and group fair decision criteria can produce counter-intuitive results, e.g. that the optimal constrained boundary may reject intuitively better candidates due to demographic imbalance in similar candidates. Both approaches can be seen as introducing individual resentment, where some individuals would have …
Partnership on AI（人工智能伙伴关系）；Minnesota State University, Mankato（明尼苏达州立大学曼卡托分校）；Sony AI（索尼人工智能）
As calls for fair and unbiased algorithmic systems increase, so too does the number of individuals working on algorithmic fairness in industry. However, these practitioners often do not have access to the demographic data they feel they need to detect bias in practice. Even with the growing variety of toolkits and strategies for working towards algorithmic fairness, they almost invariably require access to demographic attributes or proxies. We investigated this dilemma through semi-structured interviews with 38 practitioners and professionals either working in or adjacent to algorithmic fairne…
Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction
Virginia Tech（弗吉尼亚理工大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Melbourne（墨尔本大学）；University of Toronto（多伦多大学）；University of Virginia（弗吉尼亚大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
University College London（伦敦大学学院）；Duke University（杜克大学）；Apple Inc.（苹果公司）
Federated learning is an increasingly popular paradigm that enables a large number of entities to collaboratively learn better models. In this work, we study minimax group fairness in federated learning scenarios where different participating entities may only have access to a subset of the population groups during the training phase. We formally analyze how our proposed group fairness objective differs from existing federated learning fairness criteria that impose similar performance across participants instead of demographic groups. We provide an optimization algorithm -- FedMinMax -- for so…
Fair machine learning has become a significant research topic with broad societal impact. However, most fair learning methods require direct access to personal demographic data, which is increasingly restricted to use for protecting user privacy (e.g. by the EU General Data Protection Regulation). In this paper, we propose a distributed fair learning framework for protecting the privacy of demographic data. We assume this data is privately held by a third party, which can communicate with the data center (responsible for model development) without revealing the demographic information. We prop…
Are demographically invariant models and representations in medical imaging fair?
Technical University of Denmark（丹麦技术大学）；Pioneer Centre for AI（人工智能先锋中心）；Research Institute for Signals, Systems and Computational Intelligence (CONICET)（信号、系统与计算智能研究所（CONICET））；Universidad Nacional del Litoral（国立 littoral 大学）；University of Copenhagen（哥本哈根大学）；Neurobiology Research Unit, Rigshospitalet（哥本哈根大学医院神经生物学研究单元）
When dealing with sensitive data in automated data-driven decision-making, an important concern is to learn predictors with high performance towards a class label, whilst minimising for the discrimination towards any sensitive attribute, like gender or race, induced from biased data. A few hybrid tree optimisation criteria exist that combine classification performance and fairness. Although the threshold-free ROC-AUC is the standard for measuring traditional classification model performance, current fair tree classification methods mainly optimise for a fixed threshold on both the classificati…
Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China（天津理工大学计算机科学与工程学院）；Institute for Artificial Intelligence, Peking University, Beijing, China（北京大学人工智能研究院）；DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China（清华大学信息产业研究院）；National Institute of Health Data Science, Peking University, Beijing, China（北京大学健康数据科学国家研究院）
Texas A&M University（德克萨斯农工大学）；Rice University（莱斯大学）；University of Texas Health Science Center（德克萨斯大学健康科学中心）
Clinical trials are indispensable in developing new treatments, but they face obstacles in patient recruitment and retention, hindering the enrollment of necessary participants. To tackle these challenges, deep learning frameworks have been created to match patients to trials. These frameworks calculate the similarity between patients and clinical trial eligibility criteria, considering the discrepancy between inclusion and exclusion criteria. Recent studies have shown that these frameworks outperform earlier approaches. However, deep learning models may raise fairness issues in patient-trial …
There has been increasing awareness of ethical issues in machine learning, and fairness has become an important research topic. Most fairness efforts in computer vision have been focused on human sensing applications and preventing discrimination by people's physical attributes such as race, skin color or age by increasing visual representation for particular demographic groups. We argue that ML fairness efforts should extend to object recognition as well. Buildings, artwork, food and clothing are examples of the objects that define human culture. Representing these objects fairly in machine l…
In this paper, we introduce the Fairness GAN, an approach for generating a dataset that is plausibly similar to a given multimedia dataset, but is more fair with respect to protected attributes in allocative decision making. We propose a novel auxiliary classifier GAN that strives for demographic parity or equality of opportunity and show empirical results on several datasets, including the CelebFaces Attributes (CelebA) dataset, the Quick, Draw!\ dataset, and a dataset of soccer player images and the offenses they were called for. The proposed formulation is well-suited to absorbing unlabeled…
Amazon Web Services（亚马逊网络服务）；Istituto Italiano di Tecnologia（意大利理工学院）；University College London（伦敦大学学院）
Developing learning methods which do not discriminate subgroups in the population is a central goal of algorithmic fairness. One way to reach this goal is by modifying the data representation in order to meet certain fairness constraints. In this work we measure fairness according to demographic parity. This requires the probability of the possible model decisions to be independent of the sensitive information. We argue that the goal of imposing demographic parity can be substantially facilitated within a multitask learning setting. We leverage task similarities by encouraging a shared fair re…
University of Cambridge（剑桥大学）；MRC Biostatistics Unit, University of Cambridge（剑桥大学医学研究委员会生物统计组）；Sapienza University of Rome（罗马第一大学）；The Alan Turing Institute（阿兰·图灵研究所）；Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（亥姆霍兹慕尼黑中心）
While interest in the application of machine learning to improve healthcare has grown tremendously in recent years, a number of barriers prevent deployment in medical practice. A notable concern is the potential to exacerbate entrenched biases and existing health disparities in society. The area of fairness in machine learning seeks to address these issues of equity; however, appropriate approaches are context-dependent, necessitating domain-specific consideration. We focus on clinical trials, i.e., research studies conducted on humans to evaluate medical treatments. Clinical trials are a rela…
FairFedMed: Benchmarking Group Fairness in Federated Medical Imaging with FairLoRA
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Harvard University（哈佛大学）；Harvard Ophthalmology AI lab（哈佛眼科人工智能实验室）；University of Central Florida（佛罗里达中央大学）；University of Louisiana at Lafayette（路易斯安那州立大学拉斐特分校）；New York University（纽约大学）；NYUAD Center for Artificial Intelligence and Robotics（纽约大学阿布扎克人工智能与机器人中心）
Safe Fairness Guarantees Without Demographics in Classification: Spectral Uncertainty Set Perspective
Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；Predictive Analytics Lab, University of Sussex（预测分析实验室，苏塞克斯大学）；Monash University Indonesia（墨尔本大学印尼分校）；University of the Basque Country EHU（巴斯克国家大学EHU）
Demographic classification is essential in fairness assessment in recommender systems or in measuring unintended bias in online networks and voting systems. Important fields like education and politics, which often lay a foundation for the future of equality in society, need scrutiny to design policies that can better foster equality in resource distribution constrained by the unbalanced demographic distribution of people in the country. We collect three publicly available datasets to train state-of-the-art classifiers in the domain of gender and caste classification. We train the models in th…
