欺骗谷歌的观点API专为检测有毒评论而构建
Deceiving Google's Perspective API Built for Detecting Toxic Comments
摘要:
社交媒体平台提供了一个人们可以自由参与讨论的环境。不幸的是,它们也带来了一些问题,例如在线骚扰。最近,谷歌和Jigsaw启动了一个名为Perspective的项目,该项目使用机器学习来自动检测有毒语言。一个演示网站也已经启动,它允许任何人在界面中输入一个短语,并立即看到毒性评分[1]。在本文中,我们提出了一种攻击的透视有毒检测系统的基础上对抗性的例子。我们表明,对手可以巧妙地修改一个高度有毒的短语的方式,系统分配显着较低的毒性分数给it.We应用的攻击在透视网站上提供的样本短语,并表明我们可以一贯降低毒性分数的水平无毒短语。这种对抗性的例子的存在对有毒检测系统是非常有害的,并严重破坏了它们的可用性。
英文摘要:
Social media platforms provide an environment where people can freely engage in discussions. Unfortunately, they also enable several problems, such as online harassment. Recently, Google and Jigsaw started a project called Perspective, which uses machine learning to automatically detect toxic language. A demonstration website has been also launched, which allows anyone to type a phrase in the interface and instantaneously see the toxicity score [1]. In this paper, we propose an attack on the Perspective toxic detection system based on the adversarial examples. We show that an adversary can subtly modify a highly toxic phrase in a way that the system assigns significantly lower toxicity score to it. We apply the attack on the sample phrases provided in the Perspective website and show that we can consistently reduce the toxicity scores to the level of the non-toxic phrases. The existence of such adversarial examples is very harmful for toxic detection systems and seriously undermines their usability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。