arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.12337cs.LGcs.CY
收藏

识别和缓解学术推荐信中的性别线索:一个可解释性案例研究

Identifying and Mitigating Gender Cues in Academic Recommendation Letters: An Interpretability Case Study

Scheller College of Business(施谢尔商学院);Georgia Institute of Technology(佐治亚理工学院);Weinberg Institute for Cognitive Science(认知科学研究所);University of Michigan - Ann Arbor(密歇根大学安娜堡分校);School of Electrical and Computer Engineering(电气与计算机工程学院);Purdue University(普渡大学);Department of Computer Science(计算机科学系);Northwestern University(西北大学);Iowa State University(爱荷华州立大学);College of Computing(计算学院)

Charlotte S. Alexander, Shane Storks, Souradip Pal, Sayak Chakrabarty, Arushi Sharma, Mlen-Too Wesley, Bailey Russo

AI总结:

本研究探讨Transformer编码器模型和大语言模型在去除姓名和代词后,能否从匿名推荐信中推断出申请人的性别,发现存在显著的性别泄露,并通过移除隐含性别线索降低模型准确性,但性别预测仍优于随机猜测。

摘要:

推荐信(LoRs)可能隐含地携带性别化的语言模式,可能无意中影响后续决策,例如招聘和入学。本文研究了基于Transformer的编码器模型以及大语言模型(LLMs)在去除姓名和代词后,能否从提交给美国医学住院医师项目的匿名推荐信中推断出申请人的性别。使用三种模型(DistilBERT、RoBERTa和Llama 2)对匿名和去性别化的推荐信进行分类,发现显著的性别泄露,如最高68%的分类准确率。文本解释方法,如TF-IDF和SHAP,表明某些语言模式是性别强的代理,例如“情感”和“人道主义”通常与女性申请人的推荐信相关联。作为创建真正性别中性推荐信的实验,这些隐含的性别线索被移除,导致重新训练分类器时准确率下降高达5.5%和宏$F_1$得分下降2.7%。然而,申请人性别预测仍优于随机猜测。在本案例研究中,我们的发现强调:1)推荐信包含难以去除的性别识别线索,可能在决策中激活偏见;2)尽管我们的技术框架可能是实现更公平学术和职业评估的一步,但需要未来工作来探讨性别在推荐信评审中的作用。总之,我们的发现促使对现实世界学术推荐信中评估文本的上游审计,作为模型级公平干预的必要补充。

英文摘要:

Letters of recommendation (LoRs) can carry patterns of implicitly gendered language that can inadvertently influence downstream decisions, e.g. in hiring and admissions. In this work, we investigate the extent to which Transformer-based encoder models as well as Large Language Models (LLMs) can infer the gender of applicants in academic LoRs submitted to an U.S. medical-residency program after explicit identifiers like names and pronouns are de-gendered. While using three models (DistilBERT, RoBERTa, and Llama 2) to classify the gender of anonymized and de-gendered LoRs, significant gender leakage was observed as evident from up to 68% classification accuracy. Text interpretation methods, like TF-IDF and SHAP, demonstrate that certain linguistic patterns are strong proxies for gender, e.g. "emotional'' and "humanitarian'' are commonly associated with LoRs from female applicants. As an experiment in creating truly gender-neutral LoRs, these implicit gender cues were remove resulting in a drop of up to 5.5% accuracy and 2.7% macro $F_1$ score on re-training the classifiers. However, applicant gender prediction still remains better than chance. In this case study, our findings highlight that 1) LoRs contain gender-identifying cues that are hard to remove and may activate bias in decision-making and 2) while our technical framework may be a concrete step toward fairer academic and professional evaluations, future work is needed to interrogate the role that gender plays in LoR review. Taken together, our findings motivate upstream auditing of evaluative text in real-world academic letters of recommendation as a necessary complement to model-level fairness interventions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑