arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.04778stat.MLcs.LGecon.EM
收藏

离线多动作策略学习:泛化与优化

Offline Multi-Action Policy Learning: Generalization and Optimization

Stanford University(斯坦福大学)

Zhengyuan Zhou, Susan Athey, Stefan Wager

摘要:

在许多情况下,决策者希望学习一条规则或政策,将个人的可观察特征映射到行动。例如,选择要发送给消费者的报价、价格、广告或电子邮件,以及确定给患者开哪种药物的问题。虽然有越来越多的文献致力于这个问题,但大多数现有的结果都集中在数据来自随机实验的情况下,而且只有两种可能的行动,例如给患者用药或不给药。在本文中,我们研究了离线多动作策略学习问题的观测数据和政策可能需要尊重预算约束或属于一个受限制的政策类,如决策树。我们建立在有效的半参数推理的理论,以提出并实现一个政策学习算法,实现渐近最小最优遗憾。据我们所知,这是多动作设置中这种类型的第一个结果,并且它提供了比现有学习算法更大的性能改进。然后,我们考虑额外的计算挑战,在实现我们的方法的情况下,政策被限制为采取决策树的形式。我们提出了两种不同的方法,一种使用混合整数规划制定和其他使用基于树搜索的算法。

英文摘要:

In many settings, a decision-maker wishes to learn a rule, or policy, that maps from observable characteristics of an individual to an action. Examples include selecting offers, prices, advertisements, or emails to send to consumers, as well as the problem of determining which medication to prescribe to a patient. While there is a growing body of literature devoted to this problem, most existing results are focused on the case where data comes from a randomized experiment, and further, there are only two possible actions, such as giving a drug to a patient or not. In this paper, we study the offline multi-action policy learning problem with observational data and where the policy may need to respect budget constraints or belong to a restricted policy class such as decision trees. We build on the theory of efficient semi-parametric inference in order to propose and implement a policy learning algorithm that achieves asymptotically minimax-optimal regret. To the best of our knowledge, this is the first result of this type in the multi-action setup, and it provides a substantial performance improvement over the existing learning algorithms. We then consider additional computational challenges that arise in implementing our method for the case where the policy is restricted to take the form of a decision tree. We propose two different approaches, one using a mixed integer program formulation and the other using a tree-search based algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑