School of Management Fudan University(复旦大学管理学院);School of Operations Research and Information Engineering Cornell University(康奈尔大学运筹学与信息工程学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Given a traversal algorithm, cover time is the expected number of steps needed to visit all nodes in a given graph. A smaller cover time means a higher exploration efficiency of traversal algorithm. Although random walk algorithms have been studied extensively in the existing literature, there has been no cover time result for any non-Markovian method. In this work, we stand on a theoretical perspective and show that the negative feedback strategy (a count-based exploration method) is better than the naive random walk search. In particular, the former strategy can locally improve the search efficiency for an arbitrary graph. It also achieves smaller cover times for special but important graphs, including clique graphs, tree graphs, etc. Moreover, we make connections between our results and reinforcement learning literature to give new insights on why classical UCB and MCTS algorithms are so useful. Various numerical results corroborate our theoretical findings.
Adaptive KL-UCB based Bandit Algorithms for Markovian and i.i.d. Settings
Indian Institute of Technology Guwahati(印度理工学院古瓦哈提分校);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Learning in Markovian bandits with non-observable states and constrained decision epochs
IRIT, Université de Toulouse, CNRS, Toulouse INP(图卢兹大学IRIT实验室,法国国家科学研究中心,图卢兹国立理工学院);Ikerbasque-UPV/EHU, University of the Basque Country(巴斯克科学与技术联盟-巴斯克大学)
Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
Department of Electrical Engineering, Stanford University(斯坦福大学电气工程系);Industrial and Systems Engineering, Georgia Institute of Technology(佐治亚理工学院工业与系统工程系)