高风险域中偏差的检测和减小
Detecting and Reducing Bias in a High Stakes Domain
摘要：
芝加哥等城市的帮派青年有时会在社交媒体上发帖，表达他们对敌对帮派的攻击性，此前的研究表明，深度学习方法可以预测帖子中的攻击性和损失。为了解决在这个敏感的应用程序中存在偏差的可能性，我们开发了一种方法来系统地解释最先进的模型。令人惊讶的是，我们发现，它经常将其预测基于诸如“a”或“on”之类的停止词，这种方法可能会伤害没有攻击意图的社交媒体用户。为了解决这种偏见，领域专家对基本原理进行了注释，突出了解释为什么推文被贴上“侵略”标签的单词。这些新的注释使我们能够定量地衡量模型预测的合理性，并构建能够大大减少偏差的模型。我们的研究表明，在高风险的情况下，准确性本身不能保证一个良好的系统，我们需要新的评估方法。
英文摘要：
Gang-involved youth in cities such as Chicago sometimes post on social media to express their aggression towards rival gangs and previous research has demonstrated that a deep learning approach can predict aggression and loss in posts. To address the possibility of bias in this sensitive application, we developed an approach to systematically interpret the state of the art model. We found, surprisingly, that it frequently bases its predictions on stop words such as "a" or "on", an approach that could harm social media users who have no aggressive intentions. To tackle this bias, domain experts annotated the rationales, highlighting words that explain why a tweet is labeled as "aggression". These new annotations enable us to quantitatively measure how justified the model predictions are, and build models that drastically reduce bias. Our study shows that in high stake scenarios, accuracy alone cannot guarantee a good system and we need new evaluation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
