Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models. However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage. In this study, we present a fully data-driven pipeline for generating a knowledge graph (KG) of cultural knowledge and stereotypes. Our resulting KG covers 5 religious groups and 5 nationalities and can easily be extended to include more entities. Our human evaluation shows that the majority (59.2%) of non-singleton entries are coherent and complete stereotypes. We further show that performing intermediate masked language model training on the verbalized KG leads to a higher level of cultural awareness in the model and has the potential to increase classification performance on knowledge-crucial samples on a related task, i.e., hate speech detection.
The Cinderella Complex: Word Embeddings Reveal Gender Stereotypes in Movies and Books
School of Journalism and Communication, Nanjing University（南京大学新闻传播学院）；School of Systems Science, Beijing Normal University（北京师范大学系统科学学院）；School of Computing and Information, University of Pittsburgh（匹兹堡大学计算与信息学院）
MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs
Institute for Logic, Language and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）；Center for Language and Cognition, University of Groningen（格罗宁根大学语言与认知中心）
Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
Department of Computer Science, University of British Columbia（不列颠哥伦比亚大学计算机科学系）；Vector Institute for AI（人工智能矢量研究所）；Department of Linguistics, University of British Columbia（不列颠哥伦比亚大学语言学系）；Mila – Quebec AI Institute（魁北克人工智能研究所）；School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；Department of Sociology, University of British Columbia（不列颠哥伦比亚大学社会学系）
How Agents Represent Humans: Human-Directed Stereotypes in an Open Agent Social Network
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University（吉林大学知识驱动人机智能工程研究中心）；International Center of Future Science, Jilin University（吉林大学未来科学国际中心）
Language, communication and society: a gender based linguistics analysis
National Research Council of Italy（意大利国家研究委员会）；Institute of Intelligent Systems for Automation – Genoa branch（智能系统自动化研究所热那亚分部）；Institute of Electronics, Computer and Telecommunication Engineering – Genoa branch（电子、计算机与通信工程研究所热那亚分部）
CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics
Mila – Quebec AI Institute（魁北克AI研究院）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；Google Research（谷歌研究）；Google DeepMind（谷歌DeepMind）；Samsung - SAIT AI Lab（三星-SAIT人工智能实验室）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close
MBZUAI（穆罕默德·本·扎耶德人工智能大学）；The University of Tokyo（东京大学）；The University of Manchester（曼彻斯特大学）；Georgia Institute of Technology（佐治亚理工学院）；Harvard University（哈佛大学）；INSAIT, Sofia University “St. Kliment Ohridski”（INSAIT，索菲亚大学“圣克莱门特·奥赫里德斯基”）
DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis
IBM Research, San Jose, CA, USA（IBM研究院，美国加利福尼亚州圣何塞）；Dartmouth College, Hanover, NH, USA（达特茅斯学院，美国新罕布什尔州汉诺威）；IBM Research, Hursley, Winchester, UK（IBM研究院，英国赫里福德郡温切斯特）；IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA（IBM托马斯·J·沃森研究中心，美国纽约州亚当斯）；IBM Research, Sao Paulo, Brazil（IBM研究院，巴西圣保罗）
Large Language Models Discriminate Against Speakers of German Dialects
Johannes Gutenberg University Mainz（明斯特-贾格施泰因大学）；Data Science Group, University of Hamburg（汉堡大学数据科学组）；Allen Institute for AI（人工智能研究所）；University of Washington（华盛顿大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）
Multi-Reward GRPO Fine-Tuning for De-biasing Large Language Models: A Study Based on Chinese-Context Discrimination Data
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）科学与工程学院）；School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）人工智能学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society, China（深圳人工智能与机器人研究院）
Robust Bias Evaluation with FilBBQ: A Filipino Bias Benchmark for Question-Answering Language Models
School of Computer Science, University of Birmingham（伯明翰大学计算机科学学院）；Department of Information Systems and Computer Science, Ateneo de Manila University（马尼拉大学信息系统与计算机科学系）
Filipino Benchmarks for Measuring Sexist and Homophobic Bias in Multilingual Language Models from Southeast Asia
School of Computer Science, University of Birmingham（英国伯明翰大学计算机科学学院）；Department of Information Systems and Computer Science, Ateneo de Manila University（马尼拉亚托大学信息系统与计算机科学系）
