arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.11484cs.CLcs.CY
收藏

走向WinoQueer:在大语言模型中开发反酷儿偏见的基准

Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models

Information Sciences Institute(信息科学研究所);University of Southern California(南加州大学);Annenberg School for Communication and Journalism(安嫩伯格传播与新闻学院)

Virginia K. Felkner, Ho-Chun Herbert Chang, Eugene Jang, Jonathan May

摘要:

本文提出了探索性的工作是否以及在多大程度上对同性恋和跨性别者的偏见编码在大语言模型(LLM),如BERT。我们还提出了一种在下游任务中减少这些偏见的方法:根据酷儿撰写的和/或关于酷儿的数据对模型进行微调。为了测量反酷儿偏见,我们引入了一个新的基准数据集WinoQueer,该数据集以其他偏见检测基准为模型,但解决了同性恋和跨性别偏见。我们发现,BERT表现出明显的恐同偏见,但这种偏见可以通过在LGBTQ+社区成员编写的自然语言语料库上微调BERT来缓解。

英文摘要:

This paper presents exploratory work on whether and to what extent biases against queer and trans people are encoded in large language models (LLMs) such as BERT. We also propose a method for reducing these biases in downstream tasks: finetuning the models on data written by and/or about queer people. To measure anti-queer bias, we introduce a new benchmark dataset, WinoQueer, modeled after other bias-detection benchmarks but addressing homophobic and transphobic biases. We found that BERT shows significant homophobic bias, but this bias can be mostly mitigated by finetuning BERT on a natural language corpus written by members of the LGBTQ+ community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑