基于关键词的Twitter英语边缘滥用模式对边缘群体过度惩罚的理解
A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter
摘要:
有害内容检测模型往往对边缘化群体的内容具有更高的误报率。在Twitter上的边缘虐待模式的背景下,这种不成比例的惩罚造成了能见度降低的风险,边缘化社区失去了在平台上表达意见的机会。目前的算法危害缓解方法和NLP模型的偏差检测通常是非常特别的,并且容易受到人类偏见的影响。我们在本文中做出两个主要贡献。首先,我们设计了一种新的方法,它提供了一种原则性的方法来检测和测量与基于文本的模型相关的潜在危害的严重程度。其次,我们应用我们的方法来审计Twitter的英语边际滥用模型,这是用于删除放大资格的边际滥用内容。在不使用人口统计标签或方言分类器的情况下,我们仍然能够检测和衡量与边缘化社区言论过度惩罚相关问题的严重性,例如使用回收言论,反言论和身份相关术语。为了减轻相关的危害,我们尝试添加额外的真实负面示例,并发现这样做可以改善我们的公平性指标,而不会大幅降低模型性能。
英文摘要:
Harmful content detection models tend to have higher false positive rates for content from marginalized groups. In the context of marginal abuse modeling on Twitter, such disproportionate penalization poses the risk of reduced visibility, where marginalized communities lose the opportunity to voice their opinion on the platform. Current approaches to algorithmic harm mitigation, and bias detection for NLP models are often very ad hoc and subject to human bias. We make two main contributions in this paper. First, we design a novel methodology, which provides a principled approach to detecting and measuring the severity of potential harms associated with a text-based model. Second, we apply our methodology to audit Twitter's English marginal abuse model, which is used for removing amplification eligibility of marginally abusive content. Without utilizing demographic labels or dialect classifiers, we are still able to detect and measure the severity of issues related to the over-penalization of the speech of marginalized communities, such as the use of reclaimed speech, counterspeech, and identity related terms. In order to mitigate the associated harms, we experiment with adding additional true negative examples and find that doing so provides improvements to our fairness metrics without large degradations in model performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。