arXiv 2312.033792023-12-06cs.CL

用于多语言冒犯性语言识别的文本到文本模型

A Text-to-Text Model for Multilingual Offensive Language Identification

Aston University（阿斯顿大学）；George Mason University（乔治梅森大学）

Tharindu Ranasinghe, Marcos Zampieri

备注：: Accepted to Findings of IJCNLP-AACL 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出基于 T5/mT5 的编码器-解码器冒犯性语言识别模型，利用 SOLID、CCTK 训练并优化阈值，在英语及六语种基准上达到最先进性能。

摘要：

冒犯性内容在社交媒体上的无处不在，正日益引起企业和政府组织的担忧。近来，基于 transformer 的模型（如 BERT、XLNET 和 XLM-R）在检测各种形式的冒犯性内容（例如仇恨言论、网络欺凌和网络攻击）方面取得了最先进的性能。然而，这些模型中的大多数因其仅编码器架构而能力受限，该架构限制了下游任务中标签的数量和类型。为解决这些局限，本研究提出首个采用编码器-解码器架构、用于冒犯性语言识别的预训练模型，即基于文本到文本 transformer（T5）的模型，并在两个大型冒犯性语言识别数据集 SOLID 和 CCTK 上进行训练。我们研究了在 T5 再训练步骤中合并两个数据集以及在 SOLID 的半监督实例中选择最优阈值的有效性。我们的预训练 T5 模型在多个英语基准测试中优于其他为冒犯性语言检测而微调的基于 transformer 的模型，例如 fBERT 和 HateBERT。遵循类似方法，我们还使用 mT5 训练了首个用于冒犯性语言识别的多语言预训练模型，并在六种不同语言（德语、印地语、韩语、马拉地语、僧伽罗语和西班牙语）上评估其性能。结果表明，该多语言模型在上述所有数据集上均达到新的最先进水平，显示出其在多语言场景中的实用性。我们提出的基于 T5 的模型将免费向社区提供。

英文摘要：

The ubiquity of offensive content on social media is a growing cause for concern among companies and government organizations. Recently, transformer-based models such as BERT, XLNET, and XLM-R have achieved state-of-the-art performance in detecting various forms of offensive content (e.g. hate speech, cyberbullying, and cyberaggression). However, the majority of these models are limited in their capabilities due to their encoder-only architecture, which restricts the number and types of labels in downstream tasks. Addressing these limitations, this study presents the first pre-trained model with encoder-decoder architecture for offensive language identification with text-to-text transformers (T5) trained on two large offensive language identification datasets; SOLID and CCTK. We investigate the effectiveness of combining two datasets and selecting an optimal threshold in semi-supervised instances in SOLID in the T5 retraining step. Our pre-trained T5 model outperforms other transformer-based models fine-tuned for offensive language detection, such as fBERT and HateBERT, in multiple English benchmarks. Following a similar approach, we also train the first multilingual pre-trained model for offensive language identification using mT5 and evaluate its performance on a set of six different languages (German, Hindi, Korean, Marathi, Sinhala, and Spanish). The results demonstrate that this multilingual model achieves a new state-of-the-art on all the above datasets, showing its usefulness in multilingual scenarios. Our proposed T5-based models will be made freely available to the community.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文