arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1805.02400cs.CRcs.CL
收藏

保持主题:生成特定于上下文的假餐厅评论

Stay On-Topic: Generating Context-specific Fake Restaurant Reviews

Aalto University(阿尔托大学);National Institute of Information and Communications Technology(信息通信研究机构);Waseda University(早稻田大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心)

Mika Juuti, Bo Sun, Tatsuya Mori, N. Asokan

摘要:

自动生成的虚假餐馆评论对在线评论系统构成了威胁。最近的研究表明,用户很难发现隐藏在真实餐厅评论中的机器生成的虚假评论。这项工作中使用的方法(char-LSTM)有一个缺点:它很难保持在上下文中,即当它为特定目标实体生成评论时,所产生的评论可能包含与目标无关的短语,从而增加了其可检测性。在这项工作中,我们提出并评估了一种基于神经机器翻译(NMT)的更复杂的技术,我们可以使用该技术生成保持主题的评论。我们使用母语为英语的人在Amazon Mechanical Turk上测试了我们的技术的多个变体。我们证明,由最佳变体生成的评论具有几乎最佳的不可检测性(类平均F分数为47%)。我们对持怀疑态度的用户进行了一项用户研究,结果表明,与最新技术水平相比,我们的方法更频繁地逃避检测(平均逃避3.2/4 vs 1.5/4),具有统计学显著性,α = 1%(第4.3节)。我们开发了非常有效的检测工具,并在分类中达到了97%的平均F分数。虽然虚假评论在愚弄人们方面非常有效,但有效的自动检测仍然是可行的。

英文摘要:

Automatically generated fake restaurant reviews are a threat to online review systems. Recent research has shown that users have difficulties in detecting machine-generated fake reviews hiding among real restaurant reviews. The method used in this work (char-LSTM ) has one drawback: it has difficulties staying in context, i.e. when it generates a review for specific target entity, the resulting review may contain phrases that are unrelated to the target, thus increasing its detectability. In this work, we present and evaluate a more sophisticated technique based on neural machine translation (NMT) with which we can generate reviews that stay on-topic. We test multiple variants of our technique using native English speakers on Amazon Mechanical Turk. We demonstrate that reviews generated by the best variant have almost optimal undetectability (class-averaged F-score 47%). We conduct a user study with skeptical users and show that our method evades detection more frequently compared to the state-of-the-art (average evasion 3.2/4 vs 1.5/4) with statistical significance, at level α = 1% (Section 4.3). We develop very effective detection tools and reach average F-score of 97% in classifying these. Although fake reviews are very effective in fooling people, effective automatic detection is still feasible.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑