arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.14027cs.LGcs.CYcs.IR
收藏

多样性问题:维基数据中偏差测量的稳健性

Diversity matters: Robustness of bias measurements in Wikidata

Carnegie Mellon University(卡内基梅隆大学);Microsoft IDC(微软印度开发中心)

Paramita Das, Sai Keerthana Karnam, Anirban Panda, Bhanu Prakash Reddy Guda, Soumya Sarkar, Animesh Mukherjee

摘要:

随着知识图谱(KG)在各种自动化AI系统和应用中的广泛使用,确保利用它们的信息检索算法不受社会偏见的影响非常重要。以前的作品描述了在KG中持续存在的偏差,以及采用了几个指标来测量偏差。然而,这样的研究缺乏系统的探索的偏差测量的敏感性,通过不同的数据来源,或使用的嵌入算法。为了解决这一研究差距,在这项工作中,我们对知识图谱上的偏差测量进行了全面分析。首先,我们试图揭示维基数据中从七大洲选出的十三个不同人口统计数据的数据偏见。接下来,我们尝试通过两种不同的知识图谱嵌入算法- transE和complEx来展开偏差检测中的方差。我们对从13个人口统计数据中抽样的大量职业进行了广泛的实验,这些职业具有敏感属性,即,性别.我们的结果表明,KG中持续存在的固有数据偏差可以通过KG嵌入学习算法所包含的特定算法偏差来改变。此外,我们还表明,无论性别如何,最先进的KG嵌入算法的选择都会对有偏见的职业的排名产生很大影响。我们观察到,在人口统计学上,有偏见的职业的相似性是最小的,这反映了全球的社会文化差异。我们相信,这种对偏差测量管道的全面审计将提高社区的认识,同时获得与数据和算法的设计选择相关的见解,并避免流行的“一刀切”教条。

英文摘要:

With the widespread use of knowledge graphs (KG) in various automated AI systems and applications, it is very important to ensure that information retrieval algorithms leveraging them are free from societal biases. Previous works have depicted biases that persist in KGs, as well as employed several metrics for measuring the biases. However, such studies lack the systematic exploration of the sensitivity of the bias measurements, through varying sources of data, or the embedding algorithms used. To address this research gap, in this work, we present a holistic analysis of bias measurement on the knowledge graph. First, we attempt to reveal data biases that surface in Wikidata for thirteen different demographics selected from seven continents. Next, we attempt to unfold the variance in the detection of biases by two different knowledge graph embedding algorithms - TransE and ComplEx. We conduct our extensive experiments on a large number of occupations sampled from the thirteen demographics with respect to the sensitive attribute, i.e., gender. Our results show that the inherent data bias that persists in KG can be altered by specific algorithm bias as incorporated by KG embedding learning algorithms. Further, we show that the choice of the state-of-the-art KG embedding algorithm has a strong impact on the ranking of biased occupations irrespective of gender. We observe that the similarity of the biased occupations across demographics is minimal which reflects the socio-cultural differences around the globe. We believe that this full-scale audit of the bias measurement pipeline will raise awareness among the community while deriving insights related to design choices of data and algorithms both and refrain from the popular dogma of ``one-size-fits-all''.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑