arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1502.05696cs.GTcs.AIcs.LGcs.MA
收藏

众包中的批准投票和激励

Approval Voting and Incentives in Crowdsourcing

UC Berkeley(加州大学伯克利分校);Microsoft Research(微软研究院)

Nihar B. Shah, Dengyong Zhou, Yuval Peres

摘要:

对标记训练数据的需求日益增长,使得众包成为机器学习的重要组成部分。然而,众包标签的质量受到三个因素的不利影响:(1)工人不是专家;(2)工人的激励与请求者的激励不一致;(3)界面不允许工人准确地传达他们的知识,迫使他们在一组选项中做出单一选择。在本文中,我们解决这些问题,通过引入批准投票,利用专业知识的工人谁拥有部分知识的真实答案,并将其与(“严格适当”)激励兼容的补偿机制。我们严格的理论保证,我们的机制的最优性,连同一个简单的公理表征。我们还对亚马逊土耳其机器人进行了初步的实证研究,验证了我们的方法。

英文摘要:

The growing need for labeled training data has made crowdsourcing an important part of machine learning. The quality of crowdsourced labels is, however, adversely affected by three factors: (1) the workers are not experts; (2) the incentives of the workers are not aligned with those of the requesters; and (3) the interface does not allow workers to convey their knowledge accurately, by forcing them to make a single choice among a set of options. In this paper, we address these issues by introducing approval voting to utilize the expertise of workers who have partial knowledge of the true answer, and coupling it with a ("strictly proper") incentive-compatible compensation mechanism. We show rigorous theoretical guarantees of optimality of our mechanism together with a simple axiomatic characterization. We also conduct preliminary empirical studies on Amazon Mechanical Turk which validate our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑