毒性分析与情感分析方法对含有非裔美国人英语表达的语句之偏见的全面审视
A Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressions
AI总结：
本文系统研究了情感与毒性分析模型对含非裔美国人英语表达的语句的系统性偏见，通过语言、语法和语义控制特征分离 AAE 影响，证明大量使用 AAE 会显著提高毒性评分。
摘要：
语言是我们文化中一个动态的方面，在不同的技术或社群中表达时会发生变化。在线社交网络促成了不同方言的传播和演变，其中包括非裔美国人英语（AAE）。然而，这种日益增长的使用并非没有障碍。一个特别的障碍是情感分析方法（Vader、TextBlob 和 Flair）和毒性分析方法（Google 的 Perspective 和开源的 Detoxify）如何对含有 AAE 表达的语句表现出偏见。以 Google 的 Perspective 为例来理解偏见：像“All n*ggers deserve to die respectfully. The police murder us.”这样的语句，其毒性得分高于“African-Americans deserve to die respectfully. The police murder us.”。这种得分差异很可能是因为该工具无法理解“n*gger”一词的重新挪用。对这种偏见的一种解释是，AI 模型是在有限的数据集上训练的，而在训练数据中使用这样的词更可能出现在有毒语句中。虽然这看似合理，但该工具无论如何都会犯错。在此，我们研究了两个基于网络的数据集（YouTube 和 Twitter）以及两个英语口语数据集上的偏见。我们的分析表明，在大多数情境下，大多数模型都对 AAE 表现出偏见。我们通过来自 Linguistic Inquiry and Word Count（LIWC）软件的语言控制特征、通过自然语言处理（NLP）模型的词性（PoS）标注提取的语法控制特征，以及通过比较近期语言模型的句子嵌入所得到的语句语义，来分离 AAE 表达使用的影响。我们给出了一致的结果，表明大量使用 AAE 表达可能会导致说话者被认为具有显著更高的毒性，即使其谈论的主题几乎相同。我们的研究补充了那些仅关注小规模数据集和/或单一方法的类似分析。
英文摘要：
Language is a dynamic aspect of our culture that changes when expressed in different technologies/communities. Online social networks have enabled the diffusion and evolution of different dialects, including African American English (AAE). However, this increased usage is not without barriers. One particular barrier is how sentiment (Vader, TextBlob, and Flair) and toxicity (Google's Perspective and the open-source Detoxify) methods present biases towards utterances with AAE expressions. Consider Google's Perspective to understand bias. Here, an utterance such as ``All n*ggers deserve to die respectfully. The police murder us.'' it reaches a higher toxicity than ``African-Americans deserve to die respectfully. The police murder us.''. This score difference likely arises because the tool cannot understand the re-appropriation of the term ``n*gger''. One explanation for this bias is that AI models are trained on limited datasets, and using such a term in training data is more likely to appear in a toxic utterance. While this may be plausible, the tool will make mistakes regardless. Here, we study bias on two Web-based (YouTube and Twitter) datasets and two spoken English datasets. Our analysis shows how most models present biases towards AAE in most settings. We isolate the impact of AAE expression usage via linguistic control features from the Linguistic Inquiry and Word Count (LIWC) software, grammatical control features extracted via Part-of-Speech (PoS) tagging from Natural Language Processing (NLP) models, and the semantic of utterances by comparing sentence embeddings from recent language models. We present consistent results on how a heavy usage of AAE expressions may cause the speaker to be considered substantially more toxic, even when speaking about nearly the same subject. Our study complements similar analyses focusing on small datasets and/or one method only.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
