Algorithmic fairness has become an important machine learning problem, especially for mission-critical Web applications. This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Unlike existing models that target a single type of fairness, our model jointly optimizes for two fairness criteria - group fairness and counterfactual fairness - and hence makes fairer predictions at both the group and individual levels. Our model uses contrastive loss to generate embeddings that are indistinguishable for each protected group, while forcing the embeddings of counterfactual pairs to be similar. It then uses a self-knowledge distillation method to maintain the quality of representation for the downstream tasks. Extensive analysis over multiple datasets confirms the model's validity and further shows the synergy of jointly addressing two fairness criteria, suggesting the model's potential value in fair intelligent Web applications.
Floyd B. Buchanan High School（弗洛伊德·B·布坎南高中）；Massachusetts Institute of Technology（麻省理工学院）
Machine learning (ML) models are increasingly used for high-stake applications that can greatly impact people's lives. Despite their use, these models have the potential to be biased towards certain social groups on the basis of race, gender, or ethnicity. Many prior works have attempted to mitigate this "model discrimination" by updating the training data (pre-processing), altering the model learning process (in-processing), or manipulating model output (post-processing). However, these works have not yet been extended to the realm of multi-sensitive parameters and sensitive options (MSPSO), …
University of Seoul（首尔大学）；Korea University（高丽大学）；POSTECH（浦项科技大学）
Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some invariance into representation. However, adversarial learning methods are known to suffer from relatively unstable training, and this might harm the balance between fairness and predictiveness of representation. We propose a new approach, learning FAir Representation via distributional CONtrastive Variational AutoEncoder (FarconVAE), which induces the latent space to be disentangled into sensitive and nonsensitive part…
University of Virginia（弗吉尼亚大学）；UC Berkeley（加州大学伯克利分校）；UCLA（加州大学洛杉矶分校）；UIUC（伊利诺伊大学厄巴纳-香槟分校）
Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data. However, the learned representations could potentially lead to performance disparities in downstream tasks, such as increased silencing of underrepresented groups in toxicity comment classification. In light of this challenge, in this work, we study learning fair representations that satisfy a notion of fairness known as equalized odds for text classification via contrastive learning. Specifically, we first theoretically analyze the conn…
Fair Graph Neural Network with Supervised Contrastive Regularization
Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Université Toulouse 3（图卢兹第三大学）；University of the Basque Country UPV/EHU（巴斯克大学（UPV/EHU））；IKERBASQUE, Basque Foundation for Science（伊克尔巴斯克科学基金会）
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Contrastive self-supervised learning (SSL) learns an embedding space that maps similar data pairs closer and dissimilar data pairs farther apart. Despite its success, one issue has been overlooked: the fairness aspect of representations learned using contrastive SSL. Without mitigation, contrastive SSL techniques can incorporate sensitive information such as gender or race and cause potentially unfair predictions on downstream tasks. In this paper, we propose a Conditional Contrastive Learning (CCL) approach to improve the fairness of contrastive SSL methods. Our approach samples positive and …
FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learning
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；WeChat, Tencent（腾讯微信事业部）；Zhongguancun Laboratory（中关村实验室）；School of Computer, Beihang University（北京航空航天大学计算机学院）
FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data
Department of Industrial Engineering University of Central Florida（工业工程系，中央佛罗里达大学）；Department of Computer Science University of Central Florida（计算机科学系，中央佛罗里达大学）
Beijing University of Posts and Telecommunications（北京邮电大学）；BioMap（百图生科）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）
Recent studies show that graph convolutional network (GCN) often performs worse for low-degree nodes, exhibiting the so-called structural unfairness for graphs with long-tailed degree distributions prevalent in the real world. Graph contrastive learning (GCL), which marries the power of GCN and contrastive learning, has emerged as a promising self-supervised approach for learning node representations. How does GCL behave in terms of structural fairness? Surprisingly, we find that representations obtained by GCL methods are already fairer to degree bias than those learned by GCN. We theoretical…
Node representation learning has demonstrated its effectiveness for various applications on graphs. Particularly, recent developments in contrastive learning have led to promising results in unsupervised node representation learning for a number of tasks. Despite the success of graph contrastive learning and consequent growing interest, fairness is largely under-explored in the field. To this end, this study addresses fairness issues in graph contrastive learning with fairness-aware graph augmentation designs, through adaptive feature masking and edge deletion. In the study, different fairness…
Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance gains. However, recent studies have demonstrated the existence of social bias in these pretrained NLP models. Although prior works have made progress on word-level debiasing, improved sentence-level fairness of pretrained encoders still lacks exploration. In this paper, we proposed the first neural debiasing method for a pretrained sentence encoder, which transforms the pretrained encoder outputs into debiased represen…
Node representation learning has demonstrated its efficacy for various applications on graphs, which leads to increasing attention towards the area. However, fairness is a largely under-explored territory within the field, which may lead to biased results towards underrepresented groups in ensuing tasks. To this end, this work theoretically explains the sources of bias in node representations obtained via Graph Neural Networks (GNNs). Our analysis reveals that both nodal features and graph structure lead to bias in the obtained representations. Building upon the analysis, fairness-aware data a…
Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applications
Wright State University（莱特州立大学）；Cincinnati Children’s Hospital Medical Center（辛辛那提儿童医院医疗中心）；Kairos Research LLC（Kairos Research有限责任公司）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）
Trained classification models can unintentionally lead to biased representations and predictions, which can reinforce societal preconceptions and stereotypes. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. In this paper, we propose a method for mitigating bias in classifier training by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations, while instances sharing a protected attribute are forced further apart. In such a way ou…
Microsoft Research（微软研究院）；BenevolentAI（BenevolentAI公司）；University of Oxford（牛津大学）；Human Technopole（人类技术园）
Learning meaningful representations of data that can address challenges such as batch effect correction and counterfactual inference is a central problem in many domains including computational biology. Adopting a Conditional VAE framework, we show that marginal independence between the representation and a condition variable plays a key role in both of these challenges. We propose the Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment penalty defined in terms of mixtures of the variational posteriors to enforce this independence in latent space. We show that CoMP h…
FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Data
Department of Data Science and AI, Faculty of IT and Monash Energy Institute, Monash University（数据科学与人工智能系，IT学院和莫纳什能源研究所，莫纳什大学）；School of Computer Science and Technology, University of Electronic Science and Technology of China (UESTC), Chengdu, and Shenzhen Institute for Advanced Study of UESTC, Shenzhen, China（计算机科学与技术学院，电子科技大学（UESTC）成都校区，以及UESTC深圳高级研究所，深圳，中国）
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Conditional contrastive learning frameworks consider the conditional sampling procedure that constructs positive or negative data pairs conditioned on specific variables. Fair contrastive learning constructs negative pairs, for example, from the same gender (conditioning on sensitive information), which in turn reduces undesirable information from the learned representations; weakly supervised contrastive learning constructs positive pairs with similar annotative attributes (conditioning on auxiliary information), which in turn are incorporated into the representations. Although conditional co…
Indraprastha Institute of Information Technology, Delhi（德里英德拉普拉斯塔信息技术学院）；Indian Institute of Technology, Roorkee（鲁尔基印度理工学院）；International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
Retrieving facial images from attributes plays a vital role in various systems such as face recognition and suspect identification. Compared to other image retrieval tasks, facial image retrieval is more challenging due to the high subjectivity involved in describing a person's facial features. Existing methods do so by comparing specific characteristics from the user's mental image against the suggested images via high-level supervision such as using natural language. In contrast, we propose a method that uses a relatively simpler form of binary supervision by utilizing the user's feedback to…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Texas A&M University（德州农工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (we refer to it as HyperGCL). We focus on the following question: How to construct contrastive views for hypergraphs via augmentations? We provide the solutions in two folds. First, guided by domain knowledge, we fabricate two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data. Second, in search of more effective views in a data-d…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）
In the era of big data, we are often facing the challenge of data heterogeneity and the lack of label information simultaneously. In the financial domain (e.g., fraud detection), the heterogeneous data may include not only numerical data (e.g., total debt and yearly income), but also text and images (e.g., financial statement and invoice images). At the same time, the label information (e.g., fraud transactions) may be missing for building predictive models. To address these challenges, many state-of-the-art multi-view clustering methods have been proposed and achieved outstanding performance.…
Unknown University（未知大学）；Technical University of Denmark（丹麦技术大学）
Raman spectroscopy is an effective, low-cost, non-intrusive technique often used for chemical identification. Typical approaches are based on matching observations to a reference database, which requires careful preprocessing, or supervised machine learning, which requires a fairly large number of training observations from each class. We propose a new machine learning technique for Raman spectrum matching, based on contrastive representation learning, that requires no preprocessing and works with as little as a single reference spectrum from each class. On three datasets we demonstrate that o…
In this work we examine how fine-tuning impacts the fairness of contrastive Self-Supervised Learning (SSL) models. Our findings indicate that Batch Normalization (BN) statistics play a crucial role, and that updating only the BN statistics of a pre-trained SSL backbone improves its downstream fairness (36% worst subgroup, 25% mean subgroup gap). This procedure is competitive with supervised learning, while taking 4.4x less time to train and requiring only 0.35% as many parameters to be updated. Finally, inspired by recent work in supervised learning, we find that updating BN statistics and tra…
Contrastive learning is a family of self-supervised methods where a model is trained to solve a classification task constructed from unlabeled data. It has recently emerged as one of the leading learning paradigms in the absence of labels across many different domains (e.g. brain imaging, text, images). However, theoretical understanding of many aspects of training, both statistical and algorithmic, remain fairly elusive. In this work, we study the setting of time series -- more precisely, when we get data from a strong-mixing continuous-time stochastic process. We show that a properly constru…
Cross-modal contrastive learning has led the recent advances in multimodal retrieval with its simplicity and effectiveness. In this work, however, we reveal that cross-modal contrastive learning suffers from incorrect normalization of the sum retrieval probabilities of each text or video instance. Specifically, we show that many test instances are either over- or under-represented during retrieval, significantly hurting the retrieval performance. To address this problem, we propose Normalized Contrastive Learning (NCL) which utilizes the Sinkhorn-Knopp algorithm to compute the instance-wise bi…
We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. Specifically, we show that different data modalities (e.g. images and text) are embedded at arm's length in their shared representation in multi-modal models such as CLIP. Our systematic analysis demonstrates that this gap is caused by a combination of model initialization and contrastive learning optimization. In model initialization, we show empirically and theoretically that the representation of a common deep neural network is restricted to a narrow cone. As a consequence, in a mu…
Graduate School of Artificial Intelligence（人工智能研究生院）；POSTECH（浦项科技大学）
Conditional image generation is the task of generating diverse images using class label information. Although many conditional Generative Adversarial Networks (GAN) have shown realistic results, such methods consider pairwise relations between the embedding of an image and the embedding of the corresponding label (data-to-class relations) as the conditioning losses. In this paper, we propose ContraGAN that considers relations between multiple image embeddings in the same batch (data-to-data relations) as well as the data-to-class relations by using a conditional contrastive loss. The discrimin…
Global Concept Explanations for Graphs by Contrastive Learning
Institute of Theoretical Informatics, Karlsruhe Insitute of Technology, Kaiserstr. 12, 76131 Karlsruhe, Germany Institute of Nanotechnology, Karlsruhe Insitute of Technology, Kaiserstr. 12, 76131 Karlsruhe, Germany
PiCME: Pipeline for Contrastive Modality Evaluation and Encoding in the MIMIC Dataset
Department of Computer Science（计算机科学系）；Brown University（布朗大学）；Institute for Bioinformatics and Medical Informatics（生物信息学与医学信息学研究所）；University of Tübingen（图宾根大学）；Center for Computational Molecular Biology（计算分子生物学中心）
Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients
Key Laboratory of Water Big Data Technology of Ministry of Water Resources, College of Computer Science and Software Engineering, Hohai University, Nanjing, China（水利部水大数据技术重点实验室，计算机科学与软件工程学院，河海大学，南京，中国）；Department of Computer Science, City University of Hong Kong, Hong Kong, China（计算机科学系，香港城市大学，香港，中国）；State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China（新型软件技术国家重点实验室，南京大学，南京，中国）；School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine, Nanjing, China（人工智能与信息技术学院，南京中医药大学，南京，中国）
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning has become prevalent in industrial recommender systems. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe exposure bias and have a vocabulary several orders of magnitude larger than that of natural language, implying that MLE will preserve and even exacerbate the exposure bias in the long…
UiT The Arctic University of Norway（挪威北极大学）；Norwegian Computing Center（挪威计算中心）；University of Copenhagen（哥本哈根大学）
Self-supervised learning is a central component in recent approaches to deep multi-view clustering (MVC). However, we find large variations in the development of self-supervision-based methods for deep MVC, potentially slowing the progress of the field. To address this, we present DeepMVC, a unified framework for deep MVC that includes many recent methods as instances. We leverage our framework to make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning. Further, we prove that contrastive alignment can negati…
Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms
National Heart and Lung Institute, Imperial College London（伦敦帝国理工学院国家心肺研究所）；Chelsea and Westminster Hospital NHS Foundation Trust（切尔西和威斯敏斯特医院NHS基金会信托）；Zhongshan Hospital of Fudan University（复旦大学附属中山医院）；Shanghai Institute of Cardiovascular Diseases（上海市心血管病研究所）；National Clinical Research Centre for Interventional Medicine（国家介入医学临床研究中心）；Uppsala University（乌普萨拉大学）；Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center（范德堡大学医学中心临床与转化研究所）；Center for Digital Genomic Medicine, Department of Medicine, Vanderbilt University Medical Center（范德堡大学医学中心数字基因组医学中心）；Division of Cardiovascular Medicine, Vanderbilt University Medical Center（范德堡大学医学中心心血管内科）
Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learning
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China（广东省脑启发智能计算重点实验室，计算机科学与工程系，南方科技大学，深圳518055，中国）；Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen 518055, China（统计与数据科学系，南方科技大学，深圳518055，中国）
Structured Contrastive Learning for Interpretable Latent Representations
Department of Electrical and Electronic Engineering, Imperial College London（帝国理工学院伦敦电子与电气工程系）；Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford（牛津大学生物医学工程研究所）
MIFR: A Modality-Invariant and Fair Representation Framework for Skin Disease Classification
University of Dschang（德昌大学）；Université Paris-Panthéon-Assas（巴黎先贤祠-阿萨斯大学）；Université de Picardie Jules Verne（儒勒·凡尔纳皮卡第大学）；Hertie Institute for AI in Brain Health, University of Tübingen（蒂宾根大学赫蒂脑健康人工智能研究所）
We focus on better understanding the critical factors of augmentation-invariant representation learning. We revisit MoCo v2 and BYOL and try to prove the authenticity of the following assumption: different frameworks bring about representations of different characteristics even with the same pretext task. We establish the first benchmark for fair comparisons between MoCo v2 and BYOL, and observe: (i) sophisticated model configurations enable better adaptation to pre-training dataset; (ii) mismatched optimization strategies of pre-training and fine-tuning hinder model from achieving competitive…
University of Chinese Academy of Sciences（中国科学院大学）
In this paper, we introduce a self-supervised learning method to enhance the graph-level representations with the help of a set of subgraphs. For this purpose, we propose a universal framework to generate subgraphs in an auto-regressive way and then using these subgraphs to guide the learning of graph representation by Graph Neural Networks. Under this framework, we can get a comprehensive understanding of the graph structure in a learnable way. And to fully capture enough information of original graphs, we design three information aggregators: \textbf{attribute-conv}, \textbf{layer-conv} and …
