arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.06017cs.LGcs.CY
收藏

解决公平图学习数据集中的缺陷:迈向新基准

Addressing Shortcomings in Fair Graph Learning Datasets: Towards a New Benchmark

Rensselaer Polytechnic Institute(伦斯勒理工学院);The Pennsylvania State University(宾夕法尼亚州立大学);New Jersey Institute of Technology(新泽西理工学院);Michigan State University(密歇根州立大学)

Xiaowei Qian, Zhimeng Guo, Jialiang Li, Haitao Mao, Bingheng Li, Suhang Wang, Yao Ma

AI总结:

针对公平图学习数据集边信息缺失、评估不可靠的问题,本文构建含可控偏见的合成、半合成及真实数据集,并提出统一评估基准。

摘要:

公平图学习在众多实际应用中发挥着关键作用。近年来,研究者提出了许多公平图学习方法;然而,对这些方法的评估往往依赖于构造不佳的半合成数据集或质量不达标的真实世界数据集。在这种情况下,即使是基本的多层感知机(MLP),也可能在效用和公平性两方面都优于图神经网络(GNN)。在本工作中,我们说明许多数据集未能在边中提供有意义的信息,这可能会质疑在这些问题中使用图结构的必要性。为解决这些问题,我们开发并引入了一组满足广泛需求的合成、半合成和真实世界数据集。这些数据集经过精心设计,包含与模型公平评估至关重要的相关图结构和偏见信息。所提出的合成与半合成数据集提供了创建具有可控偏见参数的数据的灵活性,从而能够轻松生成具有用户定义偏见值的所需数据集。此外,我们对所提出的数据集进行了系统评估,并为公平图学习模型建立了统一的评估方法。我们在这些数据集上使用公平图学习方法得到的大量实验结果,证明了它们在基准测试这些方法性能方面的有效性。我们的数据集以及复现实验的代码可在 https://github.com/XweiQ/Benchmark-GraphFairness 获取。

英文摘要:

Fair graph learning plays a pivotal role in numerous practical applications. Recently, many fair graph learning methods have been proposed; however, their evaluation often relies on poorly constructed semi-synthetic datasets or substandard real-world datasets. In such cases, even a basic Multilayer Perceptron (MLP) can outperform Graph Neural Networks (GNNs) in both utility and fairness. In this work, we illustrate that many datasets fail to provide meaningful information in the edges, which may challenge the necessity of using graph structures in these problems. To address these issues, we develop and introduce a collection of synthetic, semi-synthetic, and real-world datasets that fulfill a broad spectrum of requirements. These datasets are thoughtfully designed to include relevant graph structures and bias information crucial for the fair evaluation of models. The proposed synthetic and semi-synthetic datasets offer the flexibility to create data with controllable bias parameters, thereby enabling the generation of desired datasets with user-defined bias values with ease. Moreover, we conduct systematic evaluations of these proposed datasets and establish a unified evaluation approach for fair graph learning models. Our extensive experimental results with fair graph learning methods across our datasets demonstrate their effectiveness in benchmarking the performance of these methods. Our datasets and the code for reproducing our experiments are available at https://github.com/XweiQ/Benchmark-GraphFairness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑