ReVar：通过减少方差抽样加强政策评估
ReVar: Strengthening Policy Evaluation via Reduced Variance Sampling
摘要：
研究了马尔可夫决策过程中用于政策评估的数据汇集问题。在策略评估中，我们被赋予一个目标策略，并被要求估计它在形式化为MDP的环境中将获得的预期累积回报。我们开发理论的最佳数据汇集类的树结构的MDP首先推导出一个Oracle数据汇集策略，使用知识的回报分布的方差。然后，我们介绍了减少方差采样（ReVar）算法，该算法近似的预言机策略时，奖励方差是未知的先验和约束其次优的预言机策略相比。最后，我们经验验证，ReVar导致政策评估的均方误差与Oracle策略相当，并显着低于简单地运行目标策略。
英文摘要：
This paper studies the problem of data collection for policy evaluation in Markov decision processes (MDPs). In policy evaluation, we are given a target policy and asked to estimate the expected cumulative reward it will obtain in an environment formalized as an MDP. We develop theory for optimal data collection within the class of tree-structured MDPs by first deriving an oracle data collection strategy that uses knowledge of the variance of the reward distributions. We then introduce the Reduced Variance Sampling (ReVar) algorithm that approximates the oracle strategy when the reward variances are unknown a priori and bound its sub-optimality compared to the oracle strategy. Finally, we empirically validate that ReVar leads to policy evaluation with mean squared error comparable to the oracle strategy and significantly lower than simply running the target policy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
