arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.09158cs.GTcs.AI
收藏

代理评分规则

Surrogate Scoring Rules

UC Santa Cruz(加州大学圣克鲁兹分校);Harvard University(哈佛大学)

Yang Liu, Juntao Wang, Yiling Chen

摘要:

严格适当的评分规则(SPSR)是激励相容的诱导随机变量的信息从战略代理人时,委托人可以奖励代理后,实现的随机变量。他们还量化了提取信息的质量,更准确的预测会获得更高的预期分数。在本文中,我们扩展了这样的评分规则设置一个主要的elephants私人概率信念,但只能访问代理的报告。我们将解决方案命名为\emph{Surrogate Scoring Rules}(SSR)。SSR建立在偏差校正步骤和出错率估计过程上,用于使用代理报告定义的参考答案。我们表明,与一个单一的信息位的先验分布的随机变量,SSR在多任务设置恢复SPSR的期望,好像有机会获得地面真相。因此,SSR的一个显著特征是,尽管缺乏地面实况,但它们量化了信息的质量,就像SPSR对设置\emph{with}地面实况所做的那样。SSR作为一种副产物,在报道中诱导\emph{dominant truthfulness}。我们的方法在理论上和经验上都得到了验证,使用从真实的人类预测者收集的数据。

英文摘要:

Strictly proper scoring rules (SPSR) are incentive compatible for eliciting information about random variables from strategic agents when the principal can reward agents after the realization of the random variables. They also quantify the quality of elicited information, with more accurate predictions receiving higher scores in expectation. In this paper, we extend such scoring rules to settings where a principal elicits private probabilistic beliefs but only has access to agents' reports. We name our solution \emph{Surrogate Scoring Rules} (SSR). SSR build on a bias correction step and an error rate estimation procedure for a reference answer defined using agents' reports. We show that, with a single bit of information about the prior distribution of the random variables, SSR in a multi-task setting recover SPSR in expectation, as if having access to the ground truth. Therefore, a salient feature of SSR is that they quantify the quality of information despite the lack of ground truth, just as SPSR do for the setting \emph{with} ground truth. As a by-product, SSR induce \emph{dominant truthfulness} in reporting. Our method is verified both theoretically and empirically using data collected from real human forecasters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑