Crowdsourcing is an effective tool for human-powered computation on many tasks challenging for computers. In this paper, we provide finite-sample exponential bounds on the error rate (in probability and in expectation) of hyperplane binary labeling rules under the Dawid-Skene crowdsourcing model. The bounds can be applied to analyze many common prediction methods, including the majority voting and weighted majority voting. These bound results could be useful for controlling the error rate and designing better algorithms. We show that the oracle Maximum A Posterior (MAP) rule approximately optimizes our upper bound on the mean error rate for any hyperplane binary labeling rule, and propose a simple data-driven weighted majority voting (WMV) rule (called one-step WMV) that attempts to approximate the oracle MAP and has a provable theoretical guarantee on the error rate. Moreover, we use simulated and real data to demonstrate that the data-driven EM-MAP rule is a good approximation to the oracle MAP rule, and to demonstrate that the mean error rate of the data-driven EM-MAP rule is also bounded by the mean error rate bound of the oracle MAP rule with estimated parameters plugging into the bound.
Enhanced Nearest Neighbor Classification for Crowdsourcing
Purdue University（普渡大学）；Binghamton University, State University of New York（纽约州立大学宾汉姆顿分校）；University of California, Los Angeles（加州大学洛杉矶分校）；Moody’s Analytics, Inc.（穆迪分析公司）
Deep learning models for predicting RNA degradation via dual crowdsourcing
Stanford University（斯坦福大学）；NVIDIA Corporation（英伟达公司）；Kaggle；Eterna Massive Open Laboratory（Eterna大规模开放实验室）；State University of New York at Buffalo（纽约州立大学布法罗分校）；High-flyer AI；DeNA；Yanfu Investments；Ghent University（根特大学）；University of Dundee（邓迪大学）；Universal Knowledge Inc.（Universal Knowledge公司）；Keyence Corporation（基恩士公司）；Texas A&M University（得克萨斯农工大学）；Rist Inc（Rist公司）；Kakao Brain；H2O；Clover Health；Afiniti；Nile University（尼罗河大学）；National Research Centre（国家研究中心）
