arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.19672cs.LGecon.EMstat.ML
收藏

退却策略学习

Policy Learning with Abstention

Ayush Sawarni, Jikai Jin, Justin Whitehouse, Vasilis Syrgkanis

AI总结:

本文提出了一种具有退却机制的策略学习方法,通过两阶段学习者识别近最优策略并构建退却规则,从而在高风险场景中提高决策安全性。

摘要:

策略学习算法广泛应用于个性化医疗和广告等领域,以开发个性化治疗方案。然而,大多数方法在预测不确定时仍强制做出决策,这在高风险场景中是危险的。我们研究了具有退却机制的策略学习,其中策略可以退回到安全的默认选项或专家。当策略退却时,它会获得一小部分附加奖励,以弥补随机猜测的价值。我们提出了一种两阶段学习者,首先识别一组近最优策略,然后从它们的分歧中构建退却规则。我们建立了当倾向性已知时的快速O(1/n)型遗憾保证,并通过双重稳健(DR)目标将这些保证扩展到倾向性未知的情况。我们进一步表明,退却是一种多功能的工具,可以直接应用于策略学习中的其他核心问题:它在不采用常见可实现性假设的情况下,通过边际条件获得改进的保证,连接到分布鲁棒策略学习,通过对小数据偏移进行对冲,以及通过确保以高概率改进基线策略来支持安全的策略改进。

英文摘要:

Policy learning algorithms are widely used in areas such as personalized medicine and advertising to develop individualized treatment regimes. However, most methods force a decision even when predictions are uncertain, which is risky in high-stakes settings. We study policy learning with abstention, where a policy may defer to a safe default or an expert. When a policy abstains, it receives a small additive reward on top of the value of a random guess. We propose a two-stage learner that first identifies a set of near-optimal policies and then constructs an abstention rule from their disagreements. We establish fast O(1/n)-type regret guarantees when propensities are known, and extend these guarantees to the unknown-propensity case via a doubly robust (DR) objective. We further show that abstention is a versatile tool with direct applications to other core problems in policy learning: it yields improved guarantees under margin conditions without the common realizability assumption, connects to distributionally robust policy learning by hedging against small data shifts, and supports safe policy improvement by ensuring improvement over a baseline policy with high probability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑