流行问答和阅读理解基准中的社会偏见
Social Bias in Popular Question-Answering Benchmarks
AI总结：
研究揭示流行问答和阅读理解基准存在社会偏见，指出其中存在人口统计学和地理偏见，并强调需改进基准创建过程以减少LLM偏见。
摘要：
问答（QA）和阅读理解（RC）基准常用于评估大语言模型（LLMs）检索和复现知识的能力。然而，我们证明了流行的QA和RC基准在代表性方面未能涵盖不同人口统计学或地区的问题。我们对30篇基准论文和20个相应的基准数据集进行了内容分析和定量分析，以了解（1）基准创建中涉及的人群，（2）基准是否表现出社会偏见，或是否对此进行了处理或防止，以及（3）创建者和标注者的 demographics 是否与内容中的偏见相关。分析的大多数基准论文提供的关于基准创建中涉及的人群的信息不足，特别是标注者。值得注意的是，仅有一篇（WinoGrande）明确报告了为解决社会代表性问题所采取的措施。此外，数据分析揭示了百科全书、常识和学术基准在广泛范围内存在性别、宗教和地理偏见。我们的工作加强了对AI评估实践的批评，并通过激励偏见推理启发式方法，揭示了偏见基准可能是LLM偏见的潜在来源。
英文摘要：
Question-answering (QA) and reading comprehension (RC) benchmarks are commonly used for assessing the capabilities of large language models (LLMs) to retrieve and reproduce knowledge. However, we demonstrate that popular QA and RC benchmarks do not cover questions about different demographics or regions in a representative way. We perform a content analysis of 30 benchmark papers and a quantitative analysis of 20 respective benchmark datasets to learn (1) who is involved in the benchmark creation, (2) whether the benchmarks exhibit social bias, or whether this is addressed or prevented, and (3) whether the demographics of the creators and annotators correspond to particular biases in the content. Most benchmark papers analyzed provide insufficient information about those involved in benchmark creation, particularly the annotators. Notably, just one (WinoGrande) explicitly reports measures taken to address social representation issues. Moreover, the data analysis revealed gender, religion, and geographic biases across a wide range of encyclopedic, commonsense, and scholarly benchmarks. Our work adds to the mounting criticism of AI evaluation practices and shines a light on biased benchmarks being a potential source of LLM bias by incentivizing biased inference heuristics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
