状态相关权和抽样分布对L1-Regularized近似线性规划逼近精度的分析
An Analysis of State-Relevance Weights and Sampling Distributions on L1-Regularized Approximate Linear Programming Approximation Accuracy
摘要：
最近对使用$L_1$正则化在使用价值函数近似中的兴趣包括Petrik等人。的介绍$L_1$ -正则化近似线性规划（RALP）。RALP在$L_1$正则化方法中是独特的，因为它使用策略外样本来近似最优价值函数。此外，它产生的策略优于以前的方法，如LSPI。RALP的价值函数近似质量的影响很大的选择状态相关权重的线性规划的目标函数，并从其中抽取样本的分布，但是，在以前的文献中没有讨论这些考虑。在本文中，我们讨论和解释的影响，在状态相关权值和抽样分布的选择近似质量，使用理论和实验说明。结果不仅提供了对这些影响的见解，而且还提供了对特别适合RALP近似的MDP类型的直观认识。
英文摘要：
Recent interest in the use of $L_1$ regularization in the use of value function approximation includes Petrik et al.'s introduction of $L_1$-Regularized Approximate Linear Programming (RALP). RALP is unique among $L_1$-regularized approaches in that it approximates the optimal value function using off-policy samples. Additionally, it produces policies which outperform those of previous methods, such as LSPI. RALP's value function approximation quality is affected heavily by the choice of state-relevance weights in the objective function of the linear program, and by the distribution from which samples are drawn; however, there has been no discussion of these considerations in the previous literature. In this paper, we discuss and explain the effects of choices in the state-relevance weights and sampling distribution on approximation quality, using both theoretical and experimental illustrations. The results provide insight not only onto these effects, but also provide intuition into the types of MDPs which are especially well suited for approximation with RALP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
