arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.12801cs.CYcs.HC
收藏

评估大语言模型对精英大学的偏见:基于人物角色的探索

Evaluation of LLMs Biases Towards Elite Universities: A Persona-Based Exploration

Shailja Gupta, Rajesh Ranjan

AI总结:

本研究通过人物角色方法,对比GPT-3.5、Gemini和Claude 3 Sonnet与LinkedIn数据,发现大语言模型对精英大学存在显著高估偏见,并提出了缓解策略。

摘要:

本研究调查了流行的大语言模型在为科技行业专业人士生成人物角色时,是否对精英大学表现出偏见。我们采用了一种新颖的基于人物角色的方法,将GPT-3.5、Gemini和Claude 3 Sonnet的教育背景预测与LinkedIn的真实数据进行比较。研究聚焦于微软、Meta和谷歌的各种职位,包括产品副总裁、工程总监和软件工程师。我们生成了432个跨三个大语言模型的人物角色,并分析了这些角色中精英大学(斯坦福大学、麻省理工学院、加州大学伯克利分校和哈佛大学)的出现频率,与LinkedIn数据进行比较。结果显示,大语言模型显著高估了精英大学,这些大学在角色中出现频率为72.45%,而实际LinkedIn数据中仅为8.56%。ChatGPT 3.5表现出最高的偏见,其次是Claude Sonnet 3,而Gemini表现最佳。这项研究强调了解决大语言模型中教育偏见的必要性,并提出了在人工智能驱动的招聘过程中减轻此类偏见的策略。

英文摘要:

This study investigates whether popular LLMs exhibit bias towards elite universities when generating personas for technology industry professionals. We employed a novel persona-based approach to compare the educational background predictions of GPT-3.5, Gemini, and Claude 3 Sonnet with actual data from LinkedIn. The study focused on various roles at Microsoft, Meta, and Google, including VP Product, Director of Engineering, and Software Engineer. We generated 432 personas across the three LLMs and analyzed the frequency of elite universities (Stanford, MIT, UC Berkeley, and Harvard) in these personas compared to LinkedIn data. Results showed that LLMs significantly overrepresented elite universities, featuring these universities 72.45% of the time, compared to only 8.56% in the actual LinkedIn data. ChatGPT 3.5 exhibited the highest bias, followed by Claude Sonnet 3, while Gemini performed best. This research highlights the need to address educational bias in LLMs and suggests strategies for mitigating such biases in AI-driven recruitment processes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑