平均报酬马尔可夫决策过程中的批策略学习
Batch Policy Learning in Average Reward Markov Decision Processes
摘要：
本文研究了无限时域马尔可夫决策过程中的批量（离线）策略学习问题。受移动健康应用程序的激励，我们专注于学习一种最大化长期平均奖励的策略。我们提出了一个双重鲁棒估计的平均回报，并表明它实现了半参数效率。进一步，我们开发了一个优化算法来计算参数化随机策略类中的最优策略。估计策略的性能通过策略类中的最优平均奖励与估计策略的平均奖励之间的差异来衡量，并且我们建立了有限样本的后悔保证。该方法的性能示出了模拟研究和分析的移动健康研究促进身体活动。
英文摘要：
We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy that maximizes the long-term average reward. We propose a doubly robust estimator for the average reward and show that it achieves semiparametric efficiency. Further we develop an optimization algorithm to compute the optimal policy in a parameterized stochastic policy class. The performance of the estimated policy is measured by the difference between the optimal average reward in the policy class and the average reward of the estimated policy and we establish a finite-sample regret guarantee. The performance of the method is illustrated by simulation studies and an analysis of a mobile health study promoting physical activity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
