HESEIA：在拉丁美洲真实学校环境中共同设计的、用于评估大型语言模型中社会偏见的社区数据集
HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin America
AI总结：
本文提出HESEIA数据集，由拉丁美洲学校师生共同设计，包含46,499个句子，用于评估大型语言模型中的交叉社会偏见，发现其包含更多现有LLM未识别的刻板印象。
摘要：
大多数用于评估大型语言模型中社会偏见的资源在开发时没有受到这些偏见影响的社区的共同设计，且很少采用参与式方法。我们引入了HESEIA，一个在专业发展课程中创建的包含46,499个句子的数据集。该课程涉及来自189所拉丁美洲学校的370名高中教师和5,370名学生。与现有基准不同，HESEIA捕捉了跨多个人口统计轴和学校科目的交叉偏见。它通过教育者的生活经验和教学专长反映了当地背景。教师使用最小对来创建表达与其学校科目和社区相关的刻板印象的句子。我们展示了该数据集在所代表的人口统计轴以及所包含的知识领域方面的多样性。我们证明，与以前的数据集相比，该数据集包含更多当前LLM无法识别的刻板印象。HESEIA可供使用，以支持基于教育社区的偏见评估。
英文摘要：
Most resources for evaluating social biases in Large Language Models are developed without co-design from the communities affected by these biases, and rarely involve participatory approaches. We introduce HESEIA, a dataset of 46,499 sentences created in a professional development course. The course involved 370 high-school teachers and 5,370 students from 189 Latin-American schools. Unlike existing benchmarks, HESEIA captures intersectional biases across multiple demographic axes and school subjects. It reflects local contexts through the lived experience and pedagogical expertise of educators. Teachers used minimal pairs to create sentences that express stereotypes relevant to their school subjects and communities. We show the dataset diversity in term of demographic axes represented and also in terms of the knowledge areas included. We demonstrate that the dataset contains more stereotypes unrecognized by current LLMs than previous datasets. HESEIA is available to support bias assessments grounded in educational communities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
