arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.16111cs.LGstat.ML
收藏

在具有生成模型的随机最短路径中的样本复杂性界

Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model

Facebook AI Research Paris(脸书人工智能研究巴黎);Inria Lille(里尔研究所);DeepMind Paris(深度思维巴黎)

Jean Tarbouriech, Matteo Pirotta, Michal Valko, Alessandro Lazaric

AI总结:

研究在随机最短路径问题中学习ε-最优策略的样本复杂性,证明在生成模型下存在最坏情况下的样本下界,并提出匹配该下界的算法。

摘要:

我们研究了在随机最短路径(SSP)问题中学习ε-最优策略的样本复杂性。我们首先推导出当学习者拥有生成模型时的样本复杂性界。我们证明存在一个最坏情况的SSP实例,具有S个状态、A个动作、最小成本c_min,以及所有状态下最优策略的最大期望成本B_∗,其中任何算法至少需要Ω(SAB_∗³/(c_minε²))个样本才能以高概率返回一个ε-最优策略。令人惊讶的是,这表明当c_min=0时SSP问题可能不可学,从而揭示了SSP的学习比有限时间 horizon 和折扣设置更困难。我们补充了这个下界,提出了在一般情况下匹配该下界的算法,并在c_min=0的情况下,只要最优策略具有有界的到达目标状态的时间,提出了匹配该下界的算法。

英文摘要:

We study the sample complexity of learning an $ε$-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access to a generative model. We show that there exists a worst-case SSP instance with $S$ states, $A$ actions, minimum cost $c_{\min}$, and maximum expected cost of the optimal policy over all states $B_{\star}$, where any algorithm requires at least $Ω(SAB_{\star}^3/(c_{\min}ε^2))$ samples to return an $ε$-optimal policy with high probability. Surprisingly, this implies that whenever $c_{\min} = 0$ an SSP problem may not be learnable, thus revealing that learning in SSPs is strictly harder than in the finite-horizon and discounted settings. We complement this lower bound with an algorithm that matches it, up to logarithmic factors, in the general case, and an algorithm that matches it up to logarithmic factors even when $c_{\min} = 0$, but only under the condition that the optimal policy has a bounded hitting time to the goal state.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑