arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.08377cs.LGstat.ML
收藏

随机最短路的隐式时域近似及有效优化算法

Implicit Finite-Horizon Approximation and Efficient Optimal Algorithms for Stochastic Shortest Path

University of Southern California(南加州大学)

Liyu Chen, Mehdi Jafarnia-Jahromi, Rahul Jain, Haipeng Luo

摘要:

我们引入了一个通用的模板,用于开发后悔最小化算法的随机最短路径(SSP)模型,只要确保某些属性,它可以实现最小最大最优后悔。我们的分析的关键是一个新的技术,称为隐式有限时域近似,近似的SSP模型的有限时域对应的分析,而没有明确的实施。使用这个模板,我们开发了两个新的算法:第一个是无模型的(据我们所知,这是文献中的第一个),并且在严格正成本下是最小最大最优的;第二个是基于模型的,并且即使在零成本状态-动作对下也是最小最大最优的,与来自[Tarbouriech等人,2021 b]。重要的是,这两种算法都允许高度稀疏的更新,这使得它们在计算上比所有现有算法都更有效。此外,两者都可以完全无参数。

英文摘要:

We introduce a generic template for developing regret minimization algorithms in the Stochastic Shortest Path (SSP) model, which achieves minimax optimal regret as long as certain properties are ensured. The key of our analysis is a new technique called implicit finite-horizon approximation, which approximates the SSP model by a finite-horizon counterpart only in the analysis without explicit implementation. Using this template, we develop two new algorithms: the first one is model-free (the first in the literature to our knowledge) and minimax optimal under strictly positive costs; the second one is model-based and minimax optimal even with zero-cost state-action pairs, matching the best existing result from [Tarbouriech et al., 2021b]. Importantly, both algorithms admit highly sparse updates, making them computationally more efficient than all existing algorithms. Moreover, both can be made completely parameter-free.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑