arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.12004cs.LGstat.MEstat.ML
收藏

CSPI-MT:针对阈值策略的校准安全策略改进与多重检验

CSPI-MT: Calibrated Safe Policy Improvement with Multiple Testing for Threshold Policies

Cornell University(康奈尔大学);Meta

Brian M Cho, Ana-Roxana Pop, Kyra Gan, Sam Corbett-Davies, Israel Nir, Ariel Evnine, Nathan Kallus

AI总结:

针对高风险阈值策略的安全改进问题,提出CSPI和CSPI-MT启发式方法,利用渐近最强安全测试与多重候选检验,在控制误差水平下显著提升安全策略检测率与改进幅度。

摘要:

在高风险环境中修改现有策略时,通常需要以高确定性确保新提出的策略优于基线(如现状)。在本研究中,我们考虑了安全策略改进问题,即只有当新策略被认为以至少预先指定的概率优于指定基线时,才采用该新策略。我们重点关注阈值策略,这是一类在经济学、医疗保健和数字广告中广泛应用的策略。现有方法依赖于可能功效不足的安全检查,并限制了寻找安全改进的机会,因此为了维持安全性,它们往往不得不退回基线。我们通过利用渐近状态下最强大的安全测试,并允许多个候选策略进行相对于基线的改进测试,从而克服了这些问题。我们证明,在对抗性环境中,即使在中等样本量下,我们的方法也能将采用劣于基线策略的比率控制在预先指定的误差水平内。我们提出了CSPI和CSPI-MT,两种用于选择截断值以最大化相对于基线策略改进的新型启发式方法。我们通过合成和外部数据集证明,我们的方法提高了安全策略的检测率和实际改进幅度,特别是在严格的安全要求和低信噪比条件下。

英文摘要:

When modifying existing policies in high-risk settings, it is often necessary to ensure with high certainty that the newly proposed policy improves upon a baseline, such as the status quo. In this work, we consider the problem of safe policy improvement, where one only adopts a new policy if it is deemed to be better than the specified baseline with at least pre-specified probability. We focus on threshold policies, a ubiquitous class of policies with applications in economics, healthcare, and digital advertising. Existing methods rely on potentially underpowered safety checks and limit the opportunities for finding safe improvements, so too often they must revert to the baseline to maintain safety. We overcome these issues by leveraging the most powerful safety test in the asymptotic regime and allowing for multiple candidates to be tested for improvement over the baseline. We show that in adversarial settings, our approach controls the rate of adopting a policy worse than the baseline to the pre-specified error level, even in moderate sample sizes. We present CSPI and CSPI-MT, two novel heuristics for selecting cutoff(s) to maximize the policy improvement from baseline. We demonstrate through both synthetic and external datasets that our approaches improve both the detection rates of safe policies and the realized improvement, particularly under stringent safety requirements and low signal-to-noise conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑