arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.01995cs.CL
收藏

基于LLM的多任务孟加拉语仇恨言论检测:类型、严重程度与目标

LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target

University of Toronto(多伦多大学);Qatar Computing Research Institute(卡塔尔计算研究院);Daffodil International University(达芙妮国际大学);Macquarie University(麦考瑞大学)

Md Arid Hasan, Firoj Alam, Md Fahad Hossain, Usman Naseem, Syed Ishtiaque Ahmed

AI总结:

针对孟加拉语仇恨言论检测多维度覆盖不足的问题,构建首个多任务人工标注数据集BanglaMultiHate,对比多类模型后发现LoRA微调LLM性能与BanglaBERT相当,文化语言适配的预训练仍很关键。

摘要:

在线社交媒体平台是日常沟通和信息获取的核心渠道。这些平台虽有积极作用,也为针对个人、组织和社群的仇恨言论、冒犯性语言及霸凌内容的传播提供了温床。此类内容会破坏网络环境的安全性、参与性与公平性,因此亟需可靠的检测系统,尤其是在审核工具匮乏的低资源语言场景中。\n在孟加拉语领域,已有研究贡献了相关资源和模型,但大多为单任务模型(如二分类的仇恨/冒犯检测),对类型、严重程度、目标等多维度信号的覆盖有限。针对这些不足,我们构建了首个多任务孟加拉语仇恨言论数据集BanglaMultiHate,这是目前规模最大的人工标注语料库之一。\n基于该资源,我们在零样本提示和LoRA微调设置下,对经典基线模型、单语预训练模型及LLM开展了全面的对照比较。实验评估了LLM在低资源场景下的适配能力,揭示了一致趋势:尽管经LoRA微调的LLM性能与BanglaBERT相当,但具备文化和语言适配性的预训练对实现稳健性能仍至关重要。我们的数据集和研究结果为低资源场景下开发符合文化背景的审核工具建立了更完善的基准。为保证可复现性,我们将公开发布该数据集及所有相关脚本。

英文摘要:

Online social media platforms are central to everyday communication and information seeking. While these platforms serve positive purposes, they also provide fertile ground for the spread of hate speech, offensive language, and bullying content targeting individuals, organizations, and communities. Such content undermines safety, participation, and equity online. Reliable detection systems are therefore needed, especially for low-resource languages where moderation tools are limited. In Bangla, prior work has contributed resources and models, but most are single-task (e.g., binary hate/offense) with limited coverage of multi-facet signals (type, severity, target). We address these gaps by introducing the first multi-task Bangla hate-speech dataset, BanglaMultiHate, one of the largest manually annotated corpus to date. Building on this resource, we conduct a comprehensive, controlled comparison spanning classical baselines, monolingual pretrained models, and LLMs under zero-shot prompting and LoRA fine-tuning. Our experiments assess LLM adaptability in a low-resource setting and reveal a consistent trend: although LoRA-tuned LLMs are competitive with BanglaBERT, culturally and linguistically grounded pretraining remains critical for robust performance. Together, our dataset and findings establish a stronger benchmark for developing culturally aligned moderation tools in low-resource contexts. For reproducibility, we will release the dataset and all related scripts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑