arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.08690cs.CLcs.AIcs.CRcs.LG
收藏

Towards Building a Robust Toxicity Predictor(构建鲁棒的毒性内容预测器)

Towards Building a Robust Toxicity Predictor

Amazon Web Services Bedrock Science(亚马逊网络服务公司Bedrock科学部门)

Dmitriy Bespalov, Sourav Bhabesh, Yi Xiang, Liutong Zhou, Yanjun Qi

AI总结:

针对毒性语言预测器鲁棒性研究不足的问题,提出新型对抗攻击方法ToxicTrap,验证了SOTA分类器的脆弱性,并证明对抗训练可提升检测器鲁棒性。

摘要:

当前自然语言处理(NLP)领域的研究对毒性语言预测器的鲁棒性关注较少,而这类系统恰恰最常被应用于对抗性场景中。本文提出了一种新型对抗攻击方法ToxicTrap,通过引入微小的词级扰动,诱导当前最优(SOTA)文本分类器将有毒文本样本误判为良性内容。ToxicTrap采用基于贪心的搜索策略,能够快速高效地生成毒性对抗样本,其设计的两种新型目标函数可同时挖掘多分类与多标签毒性语言检测器的薄弱环节。实验结果表明,现有SOTA毒性文本分类器确实易受该攻击影响,在多标签场景下攻击成功率超过98%。本文还验证了基础对抗训练及其改进版本能够提升毒性检测器的鲁棒性,甚至可抵御未见过的攻击类型。

英文摘要:

Recent NLP literature pays little attention to the robustness of toxicity language predictors, while these systems are most likely to be used in adversarial contexts. This paper presents a novel adversarial attack, \texttt{ToxicTrap}, introducing small word-level perturbations to fool SOTA text classifiers to predict toxic text samples as benign. ToxicTrap exploits greedy based search strategies to enable fast and effective generation of toxic adversarial examples. Two novel goal function designs allow ToxicTrap to identify weaknesses in both multiclass and multilabel toxic language detectors. Our empirical results show that SOTA toxicity text classifiers are indeed vulnerable to the proposed attacks, attaining over 98\% attack success rates in multilabel cases. We also show how a vanilla adversarial training and its improved version can help increase robustness of a toxicity detector even against unseen attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑