Addressing Shortcomings in Fair Graph Learning Datasets: Towards a New Benchmark
Rensselaer Polytechnic Institute(伦斯勒理工学院);The Pennsylvania State University(宾夕法尼亚州立大学);New Jersey Institute of Technology(新泽西理工学院);Michigan State University(密歇根州立大学)
Xiaowei Qian, Zhimeng Guo, Jialiang Li, Haitao Mao, Bingheng Li, Suhang Wang, Yao Ma
Fair graph learning plays a pivotal role in numerous practical applications. Recently, many fair graph learning methods have been proposed; however, their evaluation often relies on poorly constructed semi-synthetic datasets or substandard real-world datasets. In such cases, even a basic Multilayer Perceptron (MLP) can outperform Graph Neural Networks (GNNs) in both utility and fairness. In this work, we illustrate that many datasets fail to provide meaningful information in the edges, which may challenge the necessity of using graph structures in these problems. To address these issues, we develop and introduce a collection of synthetic, semi-synthetic, and real-world datasets that fulfill a broad spectrum of requirements. These datasets are thoughtfully designed to include relevant graph structures and bias information crucial for the fair evaluation of models. The proposed synthetic and semi-synthetic datasets offer the flexibility to create data with controllable bias parameters, thereby enabling the generation of desired datasets with user-defined bias values with ease. Moreover, we conduct systematic evaluations of these proposed datasets and establish a unified evaluation approach for fair graph learning models. Our extensive experimental results with fair graph learning methods across our datasets demonstrate their effectiveness in benchmarking the performance of these methods. Our datasets and the code for reproducing our experiments are available at https://github.com/XweiQ/Benchmark-GraphFairness.
LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Max Planck Institute for Security and Privacy(马克斯·普朗克安全与隐私研究所);University of Maryland(马里兰大学)
Fairness Amidst Non-IID Graph Data: A Literature Review
Florida International University(佛罗里达国际大学);Fudan University(复旦大学);UNSW Sydney(新南威尔士大学悉尼分校);CSIRO Data61(澳大利亚联邦科学与工业研究组织Data61);National Institutes of Health(美国国立卫生研究院)