The spread of online hate has become a significant problem for newspapers that host comment sections. As a result, there is growing interest in using machine learning and natural language processing for (semi-) automated abusive language detection to avoid manual comment moderation costs or having to shut down comment sections altogether. However, much of the past work on abusive language detection assumes that classifiers operate in a static language environment, despite language and news being in a state of constant flux. In this paper, we show using a new German newspaper comments dataset that the classifiers trained with naive ML techniques like a random-test train split will underperform on future data, and that a time stratified evaluation split is more appropriate. We also show that classifier performance rapidly degrades when evaluated on data from a different period than the training data. Our findings suggest that it is necessary to consider the temporal dynamics of language when developing an abusive language detection system or risk deploying a model that will quickly become defunct.
DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation
Department of Information and Decision Sciences(信息与决策科学系);University of Minnesota(明尼苏达大学);Department of Industrial Engineering and Management Sciences(工业工程与管理科学系);Northwestern University(西北大学);Fuqua School of Business(福卡商学院);Duke University(杜克大学);Department of Engineering(工程系);Carnegie Mellon University(卡内基梅隆大学)
Findings of the Shared Task on Offensive Span Identification from Code-Mixed Tamil-English Comments
Georgia Institute of Technology(佐治亚理工学院);Data Science Institute, National University of Ireland Galway(爱尔兰国立高威大学数据科学研究所);National Institute of Technology Karnataka Surathkal(卡纳塔克邦苏拉特卡尔国家理工学院);National Institute of Technology, Trichy(特里奇国家理工学院);Vellore Institute of Technology(韦洛尔理工学院);Eastern University, Sri Lanka(斯里兰卡东方大学);Indian Institute of Information Technology and Management, Kerala(喀拉拉邦印度信息技术与管理学院);Thiagarajar College of Engineering(蒂亚加拉贾工程学院)
LLMs for Argument Mining: Detection, Extraction, and Relationship Classification of pre-defined Arguments in Online Comments
School of Computing and Information Systems, The University of Melbourne(墨尔本大学计算与信息系统学院);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation
East China Normal University(华东师范大学);New York University Shanghai(上海纽约大学);University of Liverpool(利物浦大学);Singapore University of Technology and Design(新加坡科技设计大学);Eastern Institute of Technology (Ningbo)(宁波东方理工大学)
A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement
TECNALIA, Basque Research and Technology Alliance (BRTA)(TECNALIA巴斯克研究与技术联盟);University of the Basque Country (UPV/EHU)(巴斯克国家大学);Faculty of Engineering, University of Deusto(德乌斯托大学工程学院)
Detecting Sexism in German Online Newspaper Comments with Open-Source Text Embeddings (Team GDA, GermEval2024 Shared Task 1: GerMS-Detect, Subtasks 1 and 2, Closed Track)
FH Aachen University of Applied Sciences(亚琛应用技术大学);ORDIX AG(ORDIX股份公司);Utrecht University(乌得勒支大学)
FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memes
School of Computing and Information Systems, University of Melbourne, Australia(墨尔本大学计算与信息学院);Orygen, The National Centre of Excellence in Youth Mental Health, Australia(奥里根青少年心理健康国家研究中心);Centre for Youth Mental Health, University of Melbourne, Australia(墨尔本大学青少年心理健康中心);O’Donnell School of Public Health, UT Southwestern Medical Center, United States(奥唐奈公共卫生学院,西南医学中心)