arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.04902cs.DScs.LGmath.STstat.TH
收藏

一种非马尔可夫算法的覆盖时间研究

A Cover Time Study of a non-Markovian Algorithm

School of Management Fudan University(复旦大学管理学院);School of Operations Research and Information Engineering Cornell University(康奈尔大学运筹学与信息工程学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)

Guanhua Fang, Gennady Samorodnitsky, Zhiqiang Xu

AI总结:

本文首次从理论上研究非马尔可夫遍历算法的覆盖时间,证明基于计数的负反馈策略优于朴素随机游走,在团图、树图等图上覆盖时间更小,并联系强化学习解释 UCB 和 MCTS 的有效性。

摘要:

给定一个遍历算法,覆盖时间是指访问给定图中所有节点所需的期望步数。覆盖时间越小,意味着遍历算法的探索效率越高。尽管随机游走算法在现有文献中已被广泛研究,但此前没有任何关于非马尔可夫方法的覆盖时间结果。在这项工作中,我们从理论角度出发,证明负反馈策略(一种基于计数的探索方法)优于朴素随机游走搜索。特别是,前一种策略能够在任意图上局部提升搜索效率。它还在一些特殊但重要的图上实现了更小的覆盖时间,包括团图、树图等。此外,我们将所得结果与强化学习文献建立联系,为经典 UCB 和 MCTS 算法为何如此有效提供了新的见解。多种数值结果验证了我们的理论发现。

英文摘要:

Given a traversal algorithm, cover time is the expected number of steps needed to visit all nodes in a given graph. A smaller cover time means a higher exploration efficiency of traversal algorithm. Although random walk algorithms have been studied extensively in the existing literature, there has been no cover time result for any non-Markovian method. In this work, we stand on a theoretical perspective and show that the negative feedback strategy (a count-based exploration method) is better than the naive random walk search. In particular, the former strategy can locally improve the search efficiency for an arbitrary graph. It also achieves smaller cover times for special but important graphs, including clique graphs, tree graphs, etc. Moreover, we make connections between our results and reinforcement learning literature to give new insights on why classical UCB and MCTS algorithms are so useful. Various numerical results corroborate our theoretical findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑