赢家诅咒驱动数据驱动决策中的虚假承诺：难民匹配案例研究
Winner's Curse Drives False Promises in Data-Driven Decisions
AI总结：
本文揭示基于模型的政策评估方法存在赢家诅咒问题，通过难民匹配案例研究证明其在真实效果为零时仍会产生虚假收益。
摘要：
数据驱动决策中的主要挑战之一是准确的政策评估——即确保学习到的决策政策能够实现承诺的收益。一种流行的策略是基于模型的政策评估，它通过从数据中估计模型来推断反事实结果。这种策略因赢家诅咒而被认为会产生不合理的乐观估计。我们搜索了最近的数据驱动决策文献，发现过去十年中在《管理科学》上发表的55篇论文中，有53篇依赖于这种有缺陷的方法。几种常见的辩护理由被提出：（1）估计的模型是准确、稳定且校准良好的；（2）历史数据使用随机处理分配；（3）模型族是正确指定的；（4）评估方法使用样本分割。不幸的是，我们证明这些辩护中没有任何一种能避免赢家诅咒。首先，我们提供了一种理论分析，证明即使所有这些辩护都成立，赢家诅咒仍可能导致巨大的虚假报告收益。其次，我们基于最近且具有影响的难民匹配问题进行了模拟研究。我们构建了一个合成的难民匹配环境（校准以紧密匹配现实设定），但设计使得没有任何分配策略能比随机分配产生更高的预期就业率。基于模型的方法在真实效果为零的情况下仍报告了约60%的稳定收益；这些收益与文献中报告的22-75%的改进相当。我们的结果为基于模型的评估提供了强有力的反对证据。
英文摘要：
A major challenge in data-driven decision-making is accurate policy evaluation---i.e., guaranteeing that a learned decision-making policy achieves the promised benefits. A popular strategy is model-based policy evaluation, which estimates a model from data to infer counterfactual outcomes. This strategy is known to produce unwarrantedly optimistic estimates of the true benefit due to the winner's curse. We searched the recent literature on data-driven decision-making, identifying a sample of 55 papers published in Management Science in the past decade; all but two relied on this flawed methodology. Several common justifications are provided: (1) the estimated models are accurate, stable, and well-calibrated, (2) the historical data uses random treatment assignment, (3) the model family is well-specified, and (4) the evaluation methodology uses sample splitting. Unfortunately, we show that no combination of these justifications avoids the winner's curse. First, we provide a theoretical analysis demonstrating that the winner's curse can cause large, spurious reported benefits even when all these justifications hold. Second, we perform a simulation study based on the recent and consequential data-driven refugee matching problem. We construct a synthetic refugee matching environment (calibrated to closely match the real setting) but designed so that no assignment policy can improve expected employment compared to random assignment. Model-based methods report large, stable gains of around 60% even when the true effect is zero; these gains are on par with improvements of 22-75% reported in the literature. Our results provide strong evidence against model-based evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
