众包中试点研究报告的现状：对最佳实践与指南的反思
The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines
AI总结：
该研究通过对171篇相关文献的系统综述，发现众包领域试点研究存在报告不足的问题，进而提出了报告最佳实践指南及平台设计启示。
摘要：
试点研究是众包活动设计的重要基石，但在学术文献中往往仅被顺带提及。众包研究中试点研究相关细节的缺失会阻碍研究的复现与研究结果的重现，迟滞潜在的科学进展。我们针对众包与人机交互（HCI）研究交叉领域中试点研究报告的现状开展了一项系统性文献综述。我们的综述覆盖了十年间的文献，包含发表在《人类计算与众包会议（AAAI HCOMP）》论文集以及ACM数字图书馆中的171篇文章。我们发现众包研究中的试点研究（即众包试点研究）在文献中往往报告不足。诸如工人数量、工人报酬等重要细节常常未被报告。基于我们的研究结果，我们反思了当前的实践现状，并制定了一套用于在众包研究中报告众包试点研究的最佳实践指南。我们还提出了对众包平台设计的启示，并为支持众包试点研究报告给出了实用建议。
英文摘要：
Pilot studies are an essential cornerstone of the design of crowdsourcing campaigns, yet they are often only mentioned in passing in the scholarly literature. A lack of details surrounding pilot studies in crowdsourcing research hinders the replication of studies and the reproduction of findings, stalling potential scientific advances. We conducted a systematic literature review on the current state of pilot study reporting at the intersection of crowdsourcing and HCI research. Our review of ten years of literature included 171 articles published in the proceedings of the Conference on Human Computation and Crowdsourcing (AAAI HCOMP) and the ACM Digital Library. We found that pilot studies in crowdsourcing research (i.e., crowd pilot studies) are often under-reported in the literature. Important details, such as the number of workers and rewards to workers, are often not reported. On the basis of our findings, we reflect on the current state of practice and formulate a set of best practice guidelines for reporting crowd pilot studies in crowdsourcing research. We also provide implications for the design of crowdsourcing platforms and make practical suggestions for supporting crowd pilot study reporting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
