Detecting online toxicity has always been a challenge due to its inherent subjectivity. Factors such as the context, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the content can be flagged as toxic. Adoption of automated toxicity detection models in production can thus lead to a sidelining of the various groups they aim to help in the first place. It has piqued researchers' interest in examining unintended biases and their mitigation. Due to the nascent and multi-faceted nature of the work, complete literature is chaotic in its terminologies, techniques, and findings. In this paper, we put together a systematic study of the limitations and challenges of existing methods for mitigating bias in toxicity detection.
We look closely at proposed methods for evaluating and mitigating bias in toxic speech detection. To examine the limitations of existing methods, we also conduct a case study to introduce the concept of bias shift due to knowledge-based bias mitigation. The survey concludes with an overview of the critical challenges, research gaps, and future directions. While reducing toxicity on online platforms continues to be an active area of research, a systematic study of various biases and their mitigation strategies will help the research community produce robust and fair models.
Online Hate: Behavioural Dynamics and Relationship with Misinformation
Ca’ Foscari University of Venice(威尼斯大学);Jozef Stefan Institute(约瑟夫·斯特凡研究所);Jozef Stefan International Postgraduate School(约瑟夫·斯特凡国际研究生院);Sapienza University of Rome(罗马第一大学)
Defining, Understanding, and Detecting Online Toxicity: Challenges and Machine Learning Approaches
University of Duisburg-Essen, Germany(杜伊斯堡-埃森大学);Center for Advanced Internet Studies (CAIS), Germany(高级互联网研究所以(CAIS));University of Bochum, Germany(博德姆大学)
Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection
University of Saskatchewan(萨斯喀彻温大学);Singapore University of Technology and Design(新加坡科技设计大学);IIT Delhi(印度德里理工学院);IIIT-Delhi(德里英迪拉·甘地信息技术学院);DTU-Delhi(德里理工大学)