K-HATERS：带有特定目标评分的韩语仇恨言论检测语料库
K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings
AI总结：
本研究构建了首个包含约192K评论及特定目标评分的韩语仇恨言论检测语料库K-HATERS，并创新性地引入认知反思测试评估标注质量，发现低分标注者会导致模型预测产生偏见且准确率下降。
摘要：
为了对抗网络仇恨的蔓延，已有许多数据集被提出。尽管付出了这些努力，但大部分资源仍以英语为中心，主要关注显性形式的仇恨。这一研究空白要求开发高质量的多语言语料库，以涵盖更微妙的仇恨表达。本研究介绍了K-HATERS，一个新的韩语仇恨言论检测语料库，包含约192K条新闻评论，并带有特定目标的冒犯性评分。该资源是韩语中最大的冒犯性语言语料库，也是首个在三点Likert量表上提供特定目标评分的语料库，从而能够检测韩语中不同程度的冒犯性仇恨表达。我们进行了实验，通过与现有数据集进行比较，展示了所提出语料库的有效性。此外，为了解决人工标注中潜在的噪声和偏差，我们探索了一种新思路，即采用认知反思测试作为标注质量的代理指标，该测试在社会科学中广泛用于评估个体的认知能力。研究结果表明，来自测试得分最低个体的标注，往往会使检测模型对特定目标群体产生有偏见的预测，且准确性较低。本研究为仇恨言论检测和资源构建的自然语言处理（NLP）研究做出了贡献。代码和数据可在https://github.com/ssu-humane/K-HATERS获取。
英文摘要：
Numerous datasets have been proposed to combat the spread of online hate. Despite these efforts, a majority of these resources are English-centric, primarily focusing on overt forms of hate. This research gap calls for developing high-quality corpora in diverse languages that also encapsulate more subtle hate expressions. This study introduces K-HATERS, a new corpus for hate speech detection in Korean, comprising approximately 192K news comments with target-specific offensiveness ratings. This resource is the largest offensive language corpus in Korean and is the first to offer target-specific ratings on a three-point Likert scale, enabling the detection of hate expressions in Korean across varying degrees of offensiveness. We conduct experiments showing the effectiveness of the proposed corpus, including a comparison with existing datasets. Additionally, to address potential noise and bias in human annotations, we explore a novel idea of adopting the Cognitive Reflection Test, which is widely used in social science for assessing an individual's cognitive ability, as a proxy of labeling quality. Findings indicate that annotations from individuals with the lowest test scores tend to yield detection models that make biased predictions toward specific target groups and are less accurate. This study contributes to the NLP research on hate speech detection and resource construction. The code and dataset can be accessed at https://github.com/ssu-humane/K-HATERS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
