他们真的推了吗?查询事实核查网站和Politwoops以确定推文错误归因
Did They Really Tweet That? Querying Fact-Checking Sites and Politwoops to Determine Tweet Misattribution
摘要:
社交媒体帖子的截图已经成为社交媒体网站上的常见位置。虽然截图肯定是有目的的,但它们的普遍存在使得从未真正制作过的帖子的伪造截图得以传播,从而扩散了错误归因的虚假信息。出于检测这种虚假信息的动机,我们研究了开发在Web上查询推文存在证据的方法。我们开发了一种软件,可以自动使用各种服务(Google,Snopes内置搜索和路透社内置搜索)的所谓推文的主体进行搜索查询,以寻找事实核查文章和其他所谓推文的证据。我们还开发了一些工具,可以自动搜索Politwoops网站上可能由民选官员发布和删除的特定推文。此外,我们还开发了软件,从Reuters.com和Snopes.com网站上抓取事实核查文章,以便从这些网站上的任何给定文章中得出"真相评级”。为了进行评估,我们开始在实时网络上构建具有已知证据(目前只有Snopes事实检查文章)的推文的地面真实数据集,并且我们仅使用这些推文的主体进行查询,收集MRR和P@1值。这些查询表明Snopes内置搜索在大约一半的时间内有效地找到了适当的文章,MRR=0.5500和P@1=0.5333,而Google在与网站:snopes.com操作员一起使用时通常有效地找到了相关文章,MRR=0.8667和P@1=0.8667。
英文摘要:
Screenshots of social media posts have become common place on social media sites. While screenshots definitely serve a purpose, their ubiquity enables the spread of fabricated screenshots of posts that were never actually made, thereby proliferating misattribution disinformation. With the motivation of detecting this type of disinformation, we researched developing methods of querying the Web for evidence of a tweet's existence. We developed software that automatically makes search queries utilizing the body of alleged tweets to a variety of services (Google, Snopes built-in search, and Reuters built-in search) in an effort to find fact-check articles and other evidence of supposedly made tweets. We also developed tools to automatically search the site Politwoops for a particular tweet that may have been made and deleted by an elected official. In addition, we developed software to scrape fact-check articles from the sites Reuters.com and Snopes.com in order to derive a ``truth rating" from any given article from these sites. For evaluation, we began the construction of a ground truth dataset of tweets with known evidence (currently only Snopes fact-check articles) on the live web, and we gathered MRR and P@1 values based on queries made using only the bodies of those tweets. These queries showed that the Snopes built-in search was effective at finding appropriate articles about half of the time with MRR=0.5500 and P@1=0.5333, while Google when used with the site:snopes.com operator was generally effective at finding the articles in question, with MRR=0.8667 and P@1=0.8667.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。