arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.04730cs.CLcs.CY
收藏

自动分类器对男性表达的情绪检测不足

Automatic Classifiers Underdetect Emotions Expressed by Men

University of Technology Sydney(技术大学悉尼大学);University of Konstanz(康斯坦茨大学);Medical University of Vienna(维也纳医学院);University of Ulm(乌尔姆大学);Complexity Science Hub(复杂科学中心)

Ivan Smirnov, Segun T. Aroyehun, Paul Plener, David Garcia

AI总结:

本文发现自动分类器对男性表达的情绪检测存在系统性偏差,需谨慎使用以确保公平性。

摘要:

自动情感和情绪分类器的广泛应用使得确保这些工具在不同人群中可靠运行变得重要。然而,其可靠性通常通过依赖第三方注释者的基准测试来评估,而不是体验这些情绪的人本身,这可能会隐藏系统性偏见。在本文中,我们使用一个包含超过一百万条自我注释帖子的独特大规模数据集和预先注册的研究设计,研究了在414种模型和情绪相关类别组合下的情绪检测中的性别偏见。我们发现,在不同类型的自动分类器和各种基本情绪下,由男性撰写的文本的错误率始终高于由女性撰写的文本。我们量化了这种偏见如何影响下游应用的结果,并表明当前的机器学习工具,包括大型语言模型,在样本性别构成未知或变化时应谨慎使用。我们的发现表明,情感分析尚未成为解决的问题,尤其是在确保不同人口群体间模型行为公平性方面。

英文摘要:

The widespread adoption of automatic sentiment and emotion classifiers makes it important to ensure that these tools perform reliably across different populations. Yet their reliability is typically assessed using benchmarks that rely on third-party annotators rather than the individuals experiencing the emotions themselves, potentially concealing systematic biases. In this paper, we use a unique, large-scale dataset of more than one million self-annotated posts and a pre-registered research design to investigate gender biases in emotion detection across 414 combinations of models and emotion-related classes. We find that across different types of automatic classifiers and various underlying emotions, error rates are consistently higher for texts authored by men compared to those authored by women. We quantify how this bias could affect results in downstream applications and show that current machine learning tools, including large language models, should be applied with caution when the gender composition of a sample is not known or variable. Our findings demonstrate that sentiment analysis is not yet a solved problem, especially in ensuring equitable model behaviour across demographic groups.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑