arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1411.6562cs.DB
收藏

自信地评估人群

Evaluating the Crowd with Confidence

Manas Joglekar, Hector Garcia-Molina, Aditya Parameswaran

摘要:

工人质量控制是众包系统的一个关键方面;通常占用了投入众包的很大一部分时间和金钱。在这项工作中,我们设计的技术,以产生工人出错率估计的置信区间,从而使工人质量的更好的评价。我们表明,我们的技术在一系列真实世界的数据集上生成正确的置信区间,并通过使用它们来驱逐表现不佳的工人,并提供答案准确性的置信区间,从而展示了广泛的适用性。

英文摘要:

Worker quality control is a crucial aspect of crowdsourcing systems; typically occupying a large fraction of the time and money invested on crowdsourcing. In this work, we devise techniques to generate confidence intervals for worker error rate estimates, thereby enabling a better evaluation of worker quality. We show that our techniques generate correct confidence intervals on a range of real-world datasets, and demonstrate wide applicability by using them to evict poorly performing workers, and provide confidence intervals on the accuracy of the answers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑