arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07479cs.CLcs.AI
收藏

自然语言处理中的偏差检测与分类研究

A Study on Bias Detection and Classification in Natural Language Processing

Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);INESC-ID(INESC-ID研究所);Unbabel(Unbabel公司);Faculdade de Letras, Universidade de Lisboa(里斯本大学文学院)

Ana Sofia Evans, Helena Moniz, Luísa Coheur

AI总结:

针对自然语言处理领域偏差研究资源稀缺且分散的问题,研究收集公开数据集探索组合方式以优化仇恨言论检测分类模型,同时分析数据集现存问题,发现数据集组合对模型性能影响显著。

摘要:

已有研究表明,人类偏差会影响包括自然语言处理在内的多个领域的模型与算法性能。尽管近年来该现象的研究正受到关注,但可用资源仍相对稀缺,且往往聚焦于偏差的不同形式或表现。本研究的目标有两方面:1)收集公开可用的数据集,确定如何更好地组合它们,以有效训练仇恨言论检测与分类任务的模型;2)分析这些数据集的主要问题,例如数据稀缺、资源分布不均以及对非持久化数据的依赖。我们结合实验开展过程讨论了这些问题,实验结果表明不同数据集的组合会极大影响模型性能。

英文摘要:

Human biases have been shown to influence the performance of models and algorithms in various fields, including Natural Language Processing. While the study of this phenomenon is garnering focus in recent years, the available resources are still relatively scarce, often focusing on different forms or manifestations of biases. The aim of our work is twofold: 1) gather publicly-available datasets and determine how to better combine them to effectively train models in the task of hate speech detection and classification; 2) analyse the main issues with these datasets, such as scarcity, skewed resources, and reliance on non-persistent data. We discuss these issues in tandem with the development of our experiments, in which we show that the combinations of different datasets greatly impact the models' performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑