使用Amazon Mechanical Turk进行研究的隐藏成本
The Hidden Cost of Using Amazon Mechanical Turk for Research
摘要：
在这项研究中，我们调查了通过亚马逊土耳其机器人（MTurk）采购的参与者所表现出的注意力，从而发现该平台的顶级人群工作人员（被归类为“大师”的人，“批准率”为98%或更高，“批准的点击数”值为1，000或更高）中存在显著的注意力不集中。来自美国的564人参加了我们的实验。他们被要求阅读一个小插图，概述四种假设的技术产品之一，然后完成相关的调查。三种形式的注意力检查（逻辑，诚实和时间）被用来评估注意力。通过这个实验，我们确定，共有126名（22.3%）参与者未能通过三种形式的注意力检查中的至少一种，其中大多数（94人）未能通过诚实检查-其次是逻辑检查（31人）和时间检查（27人）。因此，我们确定，即使在最精英的MTurk工人中也存在显著的注意力不集中。该研究的结论是重申需要多种形式的精心制作的注意力检查，无论是否参与者的质量被认为是高根据MTurk标准，如“大师”，“批准率”和“批准的命中数”。此外，我们建议研究人员调整他们的建议，以考虑到解决参与者注意力不集中所需的努力和成本。
英文摘要：
In this study, we investigate the attentiveness exhibited by participants sourced through Amazon Mechanical Turk (MTurk), thereby discovering a significant level of inattentiveness amongst the platform's top crowd workers (those classified as 'Master', with an 'Approval Rate' of 98% or more, and a 'Number of HITS approved' value of 1,000 or more). A total of 564 individuals from the United States participated in our experiment. They were asked to read a vignette outlining one of four hypothetical technology products and then complete a related survey. Three forms of attention check (logic, honesty, and time) were used to assess attentiveness. Through this experiment we determined that a total of 126 (22.3%) participants failed at least one of the three forms of attention check, with most (94) failing the honesty check - followed by the logic check (31), and the time check (27). Thus, we established that significant levels of inattentiveness exist even among the most elite MTurk workers. The study concludes by reaffirming the need for multiple forms of carefully crafted attention checks, irrespective of whether participant quality is presumed to be high according to MTurk criteria such as 'Master', 'Approval Rate', and 'Number of HITS approved'. Furthermore, we propose that researchers adjust their proposals to account for the effort and costs required to address participant inattentiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
