arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.05045cs.HCcs.LGstat.APstat.ML
收藏

高效众包群众生成的微任务

Efficient crowdsourcing of crowd-generated microtasks

University of Vermont(佛蒙特大学)

Abigail Hotaling, James P. Bagrow

摘要:

允许群体成员为彼此提出新的微任务是将传统微任务工作的效率与人类工作者的创造力和假设生成潜力相结合的有效方法。然而,微任务提案导致越来越多的任务可能会压倒有限的众包资源。众包可以采用有效利用其资源的方法,但有效众包的算法方法通常需要已知大小的固定任务集。在本文中,我们介绍了 * 成本预测 * 作为一种手段,为众包使用高效的众包算法与越来越多的微任务。成本预测允许众包者基于新任务是否将比现有任务花费更少的完成成本来决定是从人群中引出新任务还是接收对现有任务的响应,从而在众包发生时有效地平衡资源。真实和合成众包数据的实验表明,成本预测导致准确性提高。群体生成的微任务的准确性和效率提高有望进一步利用群体的创造力和智慧,例如为机器学习应用程序生成更多信息和多样化的训练数据,以及提高用户生成内容和问答平台的性能。

英文摘要:

Allowing members of the crowd to propose novel microtasks for one another is an effective way to combine the efficiencies of traditional microtask work with the inventiveness and hypothesis generation potential of human workers. However, microtask proposal leads to a growing set of tasks that may overwhelm limited crowdsourcer resources. Crowdsourcers can employ methods to utilize their resources efficiently, but algorithmic approaches to efficient crowdsourcing generally require a fixed task set of known size. In this paper, we introduce *cost forecasting* as a means for a crowdsourcer to use efficient crowdsourcing algorithms with a growing set of microtasks. Cost forecasting allows the crowdsourcer to decide between eliciting new tasks from the crowd or receiving responses to existing tasks based on whether or not new tasks will cost less to complete than existing tasks, efficiently balancing resources as crowdsourcing occurs. Experiments with real and synthetic crowdsourcing data show that cost forecasting leads to improved accuracy. Accuracy and efficiency gains for crowd-generated microtasks hold the promise to further leverage the creativity and wisdom of the crowd, with applications such as generating more informative and diverse training data for machine learning applications and improving the performance of user-generated content and question-answering platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑