arXiv 0908.05062009-08math.OC

基于光滑线性规划的近似动态规划

Approximate Dynamic Programming via a Smoothed Linear Program

V. V. Desai, V. F. Farias, C. C. Moallemi

摘要：

提出了一种新的线性规划方法来逼近高维随机控制问题中的动态规划剩余成本函数。近似DP的LP方法通常依赖于精确动态规划的良好研究的线性规划的自然“投影”。这样的程序将注意力限制在最优剩余成本函数的下界的近似值上。我们的程序-“平滑近似线性规划”-是不同的，从这样的方法和放松限制，以适当的方式下边界近似，同时保持计算上容易处理。这样做似乎有几个优点：首先，我们证明了我们的方法所提供的最佳成本去函数的近似质量上的实质上优越的界限。第二，我们的方法上的一个具有挑战性的问题（俄罗斯方块游戏）的实验表明，该方法优于现有的LP方法（以前已被证明是有竞争力的几个ADP算法）的数量级。

英文摘要：

We present a novel linear program for the approximation of the dynamic programming cost-to-go function in high-dimensional stochastic control problems. LP approaches to approximate DP have typically relied on a natural `projection' of a well studied linear program for exact dynamic programming. Such programs restrict attention to approximations that are lower bounds to the optimal cost-to-go function. Our program--the `smoothed approximate linear program'--is distinct from such approaches and relaxes the restriction to lower bounding approximations in an appropriate fashion while remaining computationally tractable. Doing so appears to have several advantages: First, we demonstrate substantially superior bounds on the quality of approximation to the optimal cost-to-go function afforded by our approach. Second, experiments with our approach on a challenging problem (the game of Tetris) show that the approach outperforms the existing LP approach (which has previously been shown to be competitive with several ADP algorithms) by an order of magnitude.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文