搜索或审查证据可改善众包工作者的错误信息判断并减少党派偏见
Searching For or Reviewing Evidence Improves Crowdworkers' Misinformation Judgments and Reduces Partisan Bias
AI总结:
该研究通过让自由派和保守派众包工作者在判断374篇文章时搜索或审查佐证证据,发现两种方式均能减少党派分歧,其中个人搜索条件最能提升与记者评估的一致性,且16人以上众包小组的判断优于3名专家记者。
摘要:
众包工作者能否被信任来判断互联网上流传的新闻类文章是否具有误导性,还是党派倾向和经验不足会妨碍判断?任务能否以一种减少党派倾向的方式来组织?我们组建了自由派和保守派众包评分员群体,并测试了三种要求他们对374篇文章做出判断的方式。在无研究条件下,他们只被要求查看文章然后给出判断。在个人研究条件下,他们还被要求搜索佐证证据,并提供他们找到的最佳证据的链接。在集体研究条件下,他们不被要求搜索,而是审查从个人研究条件下的工作者那里收集到的链接。两种研究条件都减少了判断中的党派分歧。个人研究条件在产生与记者评估一致的结果方面最为有效。在该条件下,由十六名或更多众包工作者组成的小组的判断优于由三名专家记者组成的小组,衡量标准是与一位被留出的记者评分的一致性。
英文摘要:
Can crowd workers be trusted to judge whether news-like articles circulating on the Internet are misleading, or does partisanship and inexperience get in the way? And can the task be structured in a way that reduces partisanship? We assembled pools of both liberal and conservative crowd raters and tested three ways of asking them to make judgments about 374 articles. In a no research condition, they were just asked to view the article and then render a judgment. In an individual research condition, they were also asked to search for corroborating evidence and provide a link to the best evidence they found. In a collective research condition, they were not asked to search, but instead to review links collected from workers in the individual research condition. Both research conditions reduced partisan disagreement in judgments. The individual research condition was most effective at producing alignment with journalists' assessments. In this condition, the judgments of a panel of sixteen or more crowd workers were better than that of a panel of three expert journalists, as measured by alignment with a held out journalist's ratings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。