arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.14295math.OC
收藏

随机动态规划的信息松弛与对偶驱动算法

Information Relaxation and A Duality-Driven Algorithm for Stochastic Dynamic Programs

Nan Chen, Xiang Ma, Yanchu Liu, Wei Yu

摘要:

利用信息松弛技术,提出了一种对偶驱动的迭代方法,用于获得和改进有限时域随机动态规划问题真值的置信区间估计.我们表明,从所提出的方法产生的对偶值估计序列在原则上单调收敛到真值函数在有限数量的对偶迭代。为了克服维数灾难在各种应用中,我们还介绍了一个基于回归的蒙特卡罗算法的实现。新方法不仅可以用来评估启发式策略的质量,而且如果我们发现它们的二元性差距很大,还可以改进它们。我们得到了我们的蒙特卡罗方法的收敛速度的基函数和采样状态的数量。最后,我们证明了我们的方法在最优订单执行问题的市场摩擦和库存管理问题中存在的销售损失和提前期的有效性。这两个例子在文献中是众所周知的,难以解决的最优性。实验结果表明,我们的方法可以显着改善文献中建议的策略,并获得令人满意的性能保证的新政策。

英文摘要:

We use the technique of information relaxation to develop a duality-driven iterative approach to obtaining and improving confidence interval estimates for the true value of finite-horizon stochastic dynamic programming problems. We show that the sequence of dual value estimates yielded from the proposed approach in principle monotonically converges to the true value function in a finite number of dual iterations. Aiming to overcome the curse of dimensionality in various applications, we also introduce a regression-based Monte Carlo algorithm for implementation. The new approach can be used not only to assess the quality of heuristic policies, but also to improve them if we find that their duality gap is large. We obtain the convergence rate of our Monte Carlo method in terms of the amounts of both basis functions and the sampled states. Finally, we demonstrate the effectiveness of our method in an optimal order execution problem with market friction and in an inventory management problem in the presence of lost sale and lead time. Both examples are well known in the literature to be difficult to solve for optimality. The experiments show that our method can significantly improve the heuristics suggested in the literature and obtain new policies with a satisfactory performance guarantee.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑