arXiv 2310.139852023-10-21cs.CL

HateRephrase：使用大型语言模型实现在线帖子仇恨强度的零样本与少样本降低

HateRephrase: Zero- and Few-Shot Reduction of Hate Intensity in Online Posts using Large Language Models

University of Surrey（萨里大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）

Vibhor Agarwal, Yu Chen, Nishanth Sastry

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出HateRephrase方法，利用大型语言模型在帖子发布前改写仇恨言论以降低其强度，实验表明GPT-3.5在少样本提示下表现最佳，甚至优于人工改写。

摘要：

在当今数字时代，仇恨言论已变得无处不在。尽管已有大量研究致力于检测仇恨言论或生成反制言论来对抗仇恨观点，但这些方法仍无法完全消除仇恨言论潜在的有害社会后果——仇恨言论即使被检测到，也往往无法被删除或删除得不够彻底；而且不幸的是，仇恨言论传播迅速，通常比任何生成的反制言论都要快得多。本文研究了一种相对新颖但简单有效的方法，即在帖子发布之前就建议对潜在仇恨言论内容进行改写。我们表明，大型语言模型（LLMs）在此任务上表现良好，优于 BART-Detox 等最先进的基线方法。我们基于任务描述、仇恨定义、少样本演示和思维链开发了 4 种不同的提示，并在 LLaMA-1、LLaMA-2 chat、Vicuna 以及 OpenAI 的 GPT-3.5 等开源 LLMs 上进行了全面实验。我们提出了多种评估指标来衡量生成文本的有效性，并确保生成的文本在降低仇恨强度的同时，不会大幅改变原始文本的语义。我们发现，带有少样本演示提示的 LLMs 在生成可接受的仇恨改写文本且语义与原文相似方面效果最佳。总体而言，我们发现 GPT-3.5 在所有不同类型的提示中均优于基线模型和开源模型。我们还进行了人工评估，有趣的是，我们发现 GPT-3.5 生成的改写甚至优于数据集中人工生成的真实改写。我们还进行了详细的消融研究，以调查 LLMs 为何在此任务上表现令人满意，并进行了失败分析以理解其中的差距。

英文摘要：

Hate speech has become pervasive in today's digital age. Although there has been considerable research to detect hate speech or generate counter speech to combat hateful views, these approaches still cannot completely eliminate the potential harmful societal consequences of hate speech -- hate speech, even when detected, can often not be taken down or is often not taken down enough; and hate speech unfortunately spreads quickly, often much faster than any generated counter speech. This paper investigates a relatively new yet simple and effective approach of suggesting a rephrasing of potential hate speech content even before the post is made. We show that Large Language Models (LLMs) perform well on this task, outperforming state-of-the-art baselines such as BART-Detox. We develop 4 different prompts based on task description, hate definition, few-shot demonstrations and chain-of-thoughts for comprehensive experiments and conduct experiments on open-source LLMs such as LLaMA-1, LLaMA-2 chat, Vicuna as well as OpenAI's GPT-3.5. We propose various evaluation metrics to measure the efficacy of the generated text and ensure the generated text has reduced hate intensity without drastically changing the semantic meaning of the original text. We find that LLMs with a few-shot demonstrations prompt work the best in generating acceptable hate-rephrased text with semantic meaning similar to the original text. Overall, we find that GPT-3.5 outperforms the baseline and open-source models for all the different kinds of prompts. We also perform human evaluations and interestingly, find that the rephrasings generated by GPT-3.5 outperform even the human-generated ground-truth rephrasings in the dataset. We also conduct detailed ablation studies to investigate why LLMs work satisfactorily on this task and conduct a failure analysis to understand the gaps.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文