BanStereoSet：用于衡量孟加拉语LLMs中刻板社会偏见的数据集
BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for Bangla
AI总结：
本研究提出BanStereoSet数据集，通过本地化现有数据集构建包含1,194个句子、覆盖9类偏见的孟加拉语资源，用于评估多语言LLMs中的刻板社会偏见，分析显示模型存在显著偏见，强调了文化适应数据集的重要性。
摘要：
本研究提出了BanStereoSet，一个旨在评估多语言LLMs在孟加拉语中刻板社会偏见的数据集。为将偏见研究扩展至以英语为中心的数据集之外，我们对StereoSet、IndiBias和Kamruzzaman等人的数据集内容进行了本地化，生成了专为捕捉孟加拉语社区普遍存在的偏见而量身定制的资源。我们的BanStereoSet数据集包含1,194个句子，涵盖9类偏见：种族、职业、性别、年龄歧视、外貌、职业中的外貌、地区、种姓和宗教。该数据集不仅是衡量多语言LLMs偏见的关键工具，还促进了对不同社会类别间刻板偏见的探索，有望指导在孟加拉国背景下开发更公平的语言技术。我们使用该数据集对多个语言模型的分析表明存在显著偏见，进一步证明了文化和语言适应的数据集对于开发更公平语言技术的必要性。
英文摘要：
This study presents BanStereoSet, a dataset designed to evaluate stereotypical social biases in multilingual LLMs for the Bangla language. In an effort to extend the focus of bias research beyond English-centric datasets, we have localized the content from the StereoSet, IndiBias, and Kamruzzaman et. al.'s datasets, producing a resource tailored to capture biases prevalent within the Bangla-speaking community. Our BanStereoSet dataset consists of 1,194 sentences spanning 9 categories of bias: race, profession, gender, ageism, beauty, beauty in profession, region, caste, and religion. This dataset not only serves as a crucial tool for measuring bias in multilingual LLMs but also facilitates the exploration of stereotypical bias across different social categories, potentially guiding the development of more equitable language technologies in Bangladeshi contexts. Our analysis of several language models using this dataset indicates significant biases, reinforcing the necessity for culturally and linguistically adapted datasets to develop more equitable language technologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
