从评估视角探索大型语言模型中的孟加拉宗教方言偏见
Exploring Bengali Religious Dialect Biases in Large Language Models with Evaluation Perspectives
AI总结:
本文从评估视角探索大型语言模型在孟加拉宗教方言中的偏见,通过对ChatGPT、Gemini和Microsoft Copilot进行实验审计,揭示其在印度教和穆斯林方言上的社会偏见,为提升LLM公平性提供依据。
摘要:
尽管大型语言模型(LLM)在过去十年中产生了巨大的技术影响并支持了人类启用的应用,但它们可能会产生包含刻板印象和偏见的输出,尤其是在使用低资源语言时。在处理宗教等敏感话题时,这可能引发重大的伦理担忧。作为使LLM更加公平的途径,我们从宗教视角探索孟加拉语中的偏见,特别关注两种主要的宗教方言:印度教和穆斯林占多数的方言。我们进行了不同的实验和审计,展示使用三种常用LLM(ChatGPT、Gemini和Microsoft Copilot)对不同句子进行比较分析的结果,涉及特定词汇的印度教和穆斯林方言,并展示哪些模型捕捉到了社会偏见,哪些没有。此外,我们分析了发现并将其与潜在原因和评估视角联系起来,考虑到全球超过3亿使用者的全球影响。通过这项工作,我们希望为在LLM中创造更多公平性建立严格性,因为这些模型被广泛用作创意写作代理。
英文摘要:
While Large Language Models (LLM) have created a massive technological impact in the past decade, allowing for human-enabled applications, they can produce output that contains stereotypes and biases, especially when using low-resource languages. This can be of great ethical concern when dealing with sensitive topics such as religion. As a means toward making LLMS more fair, we explore bias from a religious perspective in Bengali, focusing specifically on two main religious dialects: Hindu and Muslim-majority dialects. Here, we perform different experiments and audit showing the comparative analysis of different sentences using three commonly used LLMs: ChatGPT, Gemini, and Microsoft Copilot, pertaining to the Hindu and Muslim dialects of specific words and showcasing which ones catch the social biases and which do not. Furthermore, we analyze our findings and relate them to potential reasons and evaluation perspectives, considering their global impact with over 300 million speakers worldwide. With this work, we hope to establish the rigor for creating more fairness in LLMs, as these are widely used as creative writing agents.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。