arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.05645cs.HC
收藏

Prototype Tasks:通过快速、迭代的任务设计改善众包结果

Prototype Tasks: Improving Crowdsourcing Results through Rapid, Iterative Task Design

Snehalkumar "Neil" S. Gaikwad, Nalin Chhibber, Vibhor Sehgal, Alipta Ballav, Catherine Mullings, Ahmed Nasser, Angela Richmond-Fuller, Aaron Gilbee, Dilrukshi G… 展开作者

Snehalkumar "Neil" S. Gaikwad, Nalin Chhibber, Vibhor Sehgal, Alipta Ballav, Catherine Mullings, Ahmed Nasser, Angela Richmond-Fuller, Aaron Gilbee, Dilrukshi Gamage, Mark Whiting, Sharon Zhou, Sekandar Matin, Senadhipathige Niranga, Shirish Goyal, Dinesh Majeti, Preethi Srinivas, Adam Ginzberg, Kamila Mananova, Karolina Ziulkoski, Jeff Regino, Tejas Sarma, Akshansh Sinha, Abhratanu Paul, Christopher Diemert, Mahesh Murag, William Dai, Manoj Pandey, Rajan Vaish, Michael Bernstein

摘要:

低质量的结果一直是微任务众包平台上的一个长期问题,赶走了请求者,并为工人的低工资辩护。到目前为止,工人们一直被指责为低质量的结果:据说他们尽可能少地努力,不注意细节,缺乏专业知识。在本文中,我们假设请求者也可能对低质量的工作负责:他们启动不明确的任务设计,甚至混淆认真的工作人员,指定边缘情况下,并忽略包括示例。我们引入了原型任务,这是一种众包策略,要求所有新的任务设计都要启动少量的样本任务。工作人员尝试这些任务,并留下反馈,使requester在发布it.We报告的现场实验中,经历原型任务迭代的任务比原来的任务设计产生更高质量的工作结果。通过这项研究,我们认为,一个简单而快速的迭代周期可以提高群众工作,我们提供的经验证据,请求者的“质量”直接影响结果的质量。

英文摘要:

Low-quality results have been a long-standing problem on microtask crowdsourcing platforms, driving away requesters and justifying low wages for workers. To date, workers have been blamed for low-quality results: they are said to make as little effort as possible, do not pay attention to detail, and lack expertise. In this paper, we hypothesize that requesters may also be responsible for low-quality work: they launch unclear task designs that confuse even earnest workers, under-specify edge cases, and neglect to include examples. We introduce prototype tasks, a crowdsourcing strategy requiring all new task designs to launch a small number of sample tasks. Workers attempt these tasks and leave feedback, enabling the re- quester to iterate on the design before publishing it. We report a field experiment in which tasks that underwent prototype task iteration produced higher-quality work results than the original task designs. With this research, we suggest that a simple and rapid iteration cycle can improve crowd work, and we provide empirical evidence that requester "quality" directly impacts result quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑