Research on hate speech has predominantly revolved around detection and interpretation from textual inputs, leaving verbal content largely unexplored. While there has been limited exploration into hate speech detection within verbal acoustic speech inputs, the aspect of interpretability has been overlooked. Therefore, we introduce a new task of explainable audio hate speech detection. Specifically, we aim to identify the precise time intervals, referred to as audio frame-level rationales, which serve as evidence for hate speech classification. Towards this end, we propose two different approaches: cascading and End-to-End (E2E). The cascading approach initially converts audio to transcripts, identifies hate speech within these transcripts, and subsequently locates the corresponding audio time frames. Conversely, the E2E approach processes audio utterances directly, which allows it to pinpoint hate speech within specific time frames. Additionally, due to the lack of explainable audio hate speech datasets that include audio frame-level rationales, we curated a synthetic audio dataset to train our models. We further validated these models on actual human speech utterances and found that the E2E approach outperforms the cascading method in terms of the audio frame Intersection over Union (IoU) metric. Furthermore, we observed that including frame-level rationales significantly enhances hate speech detection accuracy for the E2E approach.
\textbf{Disclaimer} The reader may encounter content of an offensive or hateful nature. However, given the nature of the work, this cannot be avoided.
In a hate speech detection model, we should consider two critical aspects in addition to detection performance-bias and explainability. Hate speech cannot be identified based solely on the presence of specific words: the model should be able to reason like humans and be explainable. To improve the performance concerning the two aspects, we propose Masked Rationale Prediction (MRP) as an intermediate task. MRP is a task to predict the masked human rationales-snippets of a sentence that are grounds for human judgment-by referring to surrounding tokens combined with their unmasked rationales. As …
Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）
Hate speech is a challenging issue plaguing the online social media. While better models for hate speech detection are continuously being developed, there is little research on the bias and interpretability aspects of hate speech. In this paper, we introduce HateXplain, the first benchmark hate speech dataset covering multiple aspects of the issue. Each post in our dataset is annotated from three different perspectives: the basic, commonly used 3-class classification (i.e., hate, offensive or normal), the target community (i.e., the community that has been the victim of hate speech/offensive s…
X-MuTeST: A Multilingual Benchmark for Explainable Hate Speech Detection and A Novel LLM-consulted Explanation Framework
Indian Institute of Technology Indore（印度理工学院Indore）；Indian Institute of Information Technology Dharwad（印度信息科技学院Dharwad）；Arizona State University（亚利桑那州立大学）；Indian Institute of Technology Mandi（印度理工学院Mandi）
Fraunhofer FIT（弗劳恩霍夫应用信息技术研究所）；RWTH Aachen University（亚琛工业大学）；Noakhali Science and Technology University（诺亚克利科技大学）；Begum Rokeya University（贝古姆·罗凯亚大学）；The University of Alabama（阿拉巴马大学）；National University of Ireland, Galway（爱尔兰国立高威大学）；Islamic University of Technology（伊斯兰科技大学）
The exponential growths of social media and micro-blogging sites not only provide platforms for empowering freedom of expressions and individual voices, but also enables people to express anti-social behaviour like online harassment, cyberbullying, and hate speech. Numerous works have been proposed to utilize textual data for social and anti-social behaviour analysis, by predicting the contexts mostly for highly-resourced languages like English. However, some languages are under-resourced, e.g., South Asian languages like Bengali, that lack computational resources for accurate natural language…
Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning
Center for Precision Psychiatry, Mass General Hospital Department of Psychiatry, Harvard Medical School（精准精神病学中心，麻省总医院精神病科，哈佛医学院）；D-Lab University of California, Berkeley（加州大学伯克利分校D实验室）
Boosting Accuracy and Interpretability in Multilingual Hate Speech Detection Through Layer Freezing and Explainable AI
Department of Computer Science（计算机科学系）；University of South Carolina（南卡罗来纳大学）；Department of Civil, Environmental, and Ocean Engineering（土木、环境与海洋工程系）；Stevens Institute of Technology（史蒂文斯理工学院）；School of Computing（计算学院）；University of Nebraska–Lincoln（内布拉斯加大学林肯分校）；Bowling Green State University（鲍灵格林州立大学）
The rise of hate speech on online platforms has led to an urgent need for effective content moderation. However, the subjective and multi-faceted nature of hateful online content, including implicit hate speech, poses significant challenges to human moderators and content moderation systems. To address this issue, we developed ToxVis, a visually interactive and explainable tool for classifying hate speech into three categories: implicit, explicit, and non-hateful. We fine-tuned two transformer-based models using RoBERTa, XLNET, and GPT-3 and used deep learning interpretation techniques to prov…
AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）
TANDEM: Temporal-Aware Neural Detection for Multimodal Hate Speech
Nature-Inspired Computing & Engineering, University of Surrey（Surrey大学自然启发计算与工程系）；Surrey Centre for Cyber Security, University of Surrey（Surrey大学网络安全中心）
School of Computer Sciences, NISER（国家科学教育与研究院计算机科学学院）；Homi Bhabha National Institute（霍米·巴巴国家研究所）
Hate speech, quite common in the age of social media, at times harmless but can also cause mental trauma to someone or even riots in communities. Image of a religious symbol with derogatory comment or video of a man abusing a particular community, all become hate speech with its every modality (such as text, image, and audio) contributing towards it. Models based on a particular modality of hate speech post on social media are not useful, rather, we need models like multi-modal fusion models that consider both image and text while classifying hate speech. Text-image fusion models are heavily p…
Hate Speech takes many forms to target communities with derogatory comments, and takes humanity a step back in societal progress. HateXplain is a recently published and first dataset to use annotated spans in the form of rationales, along with speech classification categories and targeted communities to make the classification more humanlike, explainable, accurate and less biased. We tune BERT to perform this task in the form of rationales and class prediction, and compare our performance on different metrics spanning across accuracy, explainability and bias. Our novelty is threefold. Firstly,…
We present a novel feature attribution method for explaining text classifiers, and analyze it in the context of hate speech detection. Although feature attribution models usually provide a single importance score for each token, we instead provide two complementary and theoretically-grounded scores -- necessity and sufficiency -- resulting in more informative explanations. We propose a transparent method that calculates these values by generating explicit perturbations of the input text, allowing the importance scores themselves to be explainable. We employ our method to explain the prediction…
From Large-scale Audio Tagging to Real-Time Explainable Emergency Vehicle Sirens Detection
Department of Information Engineering, Computer Science and Mathematics (DISIM), University of L’Aquila（信息工程、计算机科学与数学系（DISIM），拉奎拉大学）；National Inter-University Consortium for Telecommunications (CNIT), University of L’Aquila（电信国家大学间联合体（CNIT），拉奎拉大学）
We conduct relatively extensive investigations of automatic hate speech (HS) detection using different state-of-the-art (SoTA) baselines over 11 subtasks of 6 different datasets. Our motivation is to determine which of the recent SoTA models is best for automatic hate speech detection and what advantage methods like data augmentation and ensemble may have on the best model, if any. We carry out 6 cross-task investigations. We achieve new SoTA on two subtasks - macro F1 scores of 91.73% and 53.21% for subtasks A and B of the HASOC 2020 dataset, where previous SoTA are 51.52% and 26.52%, respect…
We investigate the performance of a state-of-the art (SoTA) architecture T5 (available on the SuperGLUE) and compare with it 3 other previous SoTA architectures across 5 different tasks from 2 relatively diverse datasets. The datasets are diverse in terms of the number and types of tasks they have. To improve performance, we augment the training data by using an autoregressive model. We achieve near-SoTA results on a couple of the tasks - macro F1 scores of 81.66% for task A of the OLID 2019 dataset and 82.54% for task A of the hate speech and offensive content (HASOC) 2021 dataset, where SoTA…
HateMirage: An Explainable Multi-Dimensional Dataset for Decoding Faux Hate and Subtle Online Abuse
Indian Institute of Information Technology Dharwad（印度信息技术学院达尔瓦德）；Manipal Institute of Technology, Manipal Academy of Higher Education（曼普及高等教育学院技术学院）；Indraprastha Institute of Information Technology Delhi（印度普拉斯塔信息技术学院德里）
METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark
Institute of Artificial Intelligence (TeleAI), China Telecom（电信人工智能研究院）；Institute of Artificial Intelligence and Robotics(IAIR), Xi’an Jiaotong University（人工智能与机器人研究院）；Advanced Technique of Artificial Intelligence(ATAI), Chongqing University of Technology（人工智能先进技术研究院）
Metamorphic Testing for Audio Content Moderation Software
School of Information, Renmin University of China（中国人民大学信息学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Department of Electrical Engineering, City University of Hong Kong（香港城市大学电子工程系）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科学大学计算机科学与工程系）
XEmoGPT: An Explainable Multimodal Emotion Recognition Framework with Cue-Level Perception and Reasoning
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；th Research Institute, China Electronics Technology Group Corporation（中国电子科技集团第五十四研究所）
BenSParX: A Robust Explainable Machine Learning Framework for Parkinson's Disease Detection from Bengali Conversational Speech
Chittagong University of Engineering and Technology（吉大港工程技术大学）；East Delta University（东三角洲大学）；Charles Sturt University（查尔斯特大学）；Bangabandhu Sheikh Mujib Medical University（班加班杜·谢赫·穆吉布医科大学）
GSRM: Generative Speech Reward Model for Speech RLHF
Meta Superintelligence Labs（Meta超智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Arizona State University（亚利桑那州立大学）；University of Southern California（南加州大学）
Unveiling Hidden Factors: Explainable AI for Feature Boosting in Speech Emotion Recognition
University of Québec (TÉLUQ)（魁北克大学（TÉLUQ））；Concordia University（康考迪亚大学）；Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程学院）；University of Québec at Montréal（蒙特利尔魁北克大学）；Centre for Research and Intervention on Suicide, Ethical Issues and End-of-Life Practices（自杀、伦理问题与临终实践研究与干预中心）
