用适当评分规则激励诚实的执行性预测
Incentivizing honest performative predictions with proper scoring rules
AI总结:
本文研究预测可影响结果时如何设计评分规则以激励诚实报告,证明二元预测下可使最优报告任意接近不动点,而多结果预测则不可能,并通过数值模拟验证了误差界。
摘要:
适当评分规则激励专家准确报告信念,前提是假设预测不会影响结果。我们放宽这一假设,研究当预测具有执行性时的激励问题,即预测可以影响预测结果的情形,例如公开发布关于股市的预测。如果某个预测在做出后准确反映了专家的信念,我们称该预测为不动点。我们表明,在这一设定下,最大化期望得分的报告通常不能反映专家的信念,并给出了此类报告不准确性的界。我们证明,对于二元预测,如果专家预测对结果的影响是有界的,则可以定义评分规则,使得最优报告任意接近不动点。然而,对于超过两个结果的预测,这是不可能的。我们还在一个玩具设定中进行了数值模拟,表明我们的界在某些情形下是紧的,并且预测误差通常很大(大于5-10%)。最后,我们讨论了最优性的替代概念,包括执行稳定性,并证明它们会激励报告不动点。
英文摘要:
Proper scoring rules incentivize experts to accurately report beliefs, assuming predictions cannot influence outcomes. We relax this assumption and investigate incentives when predictions are performative, i.e., when they can influence the outcome of the prediction, such as when making public predictions about the stock market. We say a prediction is a fixed point if it accurately reflects the expert's beliefs after that prediction has been made. We show that in this setting, reports maximizing expected score generally do not reflect an expert's beliefs, and we give bounds on the inaccuracy of such reports. We show that, for binary predictions, if the influence of the expert's prediction on outcomes is bounded, it is possible to define scoring rules under which optimal reports are arbitrarily close to fixed points. However, this is impossible for predictions over more than two outcomes. We also perform numerical simulations in a toy setting, showing that our bounds are tight in some situations and that prediction error is often substantial (greater than 5-10%). Lastly, we discuss alternative notions of optimality, including performative stability, and show that they incentivize reporting fixed points.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。