With the widespread use of knowledge graphs (KG) in various automated AI systems and applications, it is very important to ensure that information retrieval algorithms leveraging them are free from societal biases. Previous works have depicted biases that persist in KGs, as well as employed several metrics for measuring the biases. However, such studies lack the systematic exploration of the sensitivity of the bias measurements, through varying sources of data, or the embedding algorithms used. To address this research gap, in this work, we present a holistic analysis of bias measurement on the knowledge graph. First, we attempt to reveal data biases that surface in Wikidata for thirteen different demographics selected from seven continents. Next, we attempt to unfold the variance in the detection of biases by two different knowledge graph embedding algorithms - TransE and ComplEx. We conduct our extensive experiments on a large number of occupations sampled from the thirteen demographics with respect to the sensitive attribute, i.e., gender. Our results show that the inherent data bias that persists in KG can be altered by specific algorithm bias as incorporated by KG embedding learning algorithms. Further, we show that the choice of the state-of-the-art KG embedding algorithm has a strong impact on the ranking of biased occupations irrespective of gender. We observe that the similarity of the biased occupations across demographics is minimal which reflects the socio-cultural differences around the globe. We believe that this full-scale audit of the bias measurement pipeline will raise awareness among the community while deriving insights related to design choices of data and algorithms both and refrain from the popular dogma of ``one-size-fits-all''.
Wisdom from Diversity: Bias Mitigation Through Hybrid Human-LLM Crowds
Machine Learning Group, Université Libre de Bruxelles(布鲁塞尔自由大学机器学习组);AI Lab, Vrije Universiteit Brussel(布鲁塞尔自由大学人工智能实验室);FARI, AI for the Common-Good Institute, ULB-VUB(ULB-VUB共同利益人工智能研究所);Center for Human-Compatible AI, UC Berkeley(伯克利人类兼容人工智能中心)
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Georgia Institute of Technology(佐治亚理工学院);University of Washington(华盛顿大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Google Research(谷歌研究院)
The impact of data set similarity and diversity on transfer learning success in time series forecasting
Fraunhofer Institute for Integrated Circuits IIS(弗劳恩霍夫集成电路研究所);Fraunhofer Center for Applied Research on Supply Chain Services SCS(弗劳恩霍夫供应链服务应用研究中心);Christian-Albrechts-Universität zu Kiel(基尔基督教阿尔布雷希特大学);Otto-Friedrich-Universität Bamberg(班贝格奥托-弗里德里希大学);Vives University of Applied Science(维维斯应用科技大学);University of Haute-Alsace(上阿尔萨斯大学);Monash University(莫纳什大学)