The on-line shortest path problem under partial monitoring
Computer and Automation Research Institute of the Hungarian Academy of Sciences(匈牙利科学院计算机与自动化研究所);Queen’s University(女王大学);ICREA(加泰罗尼亚研究与高级研究院);Universitat Pompeu Fabra(庞培法布拉大学);Budapest University of Technology and Economics(布达佩斯技术与经济大学)
The on-line shortest path problem is considered under various models of partial monitoring. Given a weighted directed acyclic graph whose edge weights can change in an arbitrary (adversarial) way, a decision maker has to choose in each round of a game a path between two distinguished vertices such that the loss of the chosen path (defined as the sum of the weights of its composing edges) be as small as possible. In a setting generalizing the multi-armed bandit problem, after choosing a path, the decision maker learns only the weights of those edges that belong to the chosen path. For this problem, an algorithm is given whose average cumulative loss in n rounds exceeds that of the best path, matched off-line to the entire sequence of the edge weights, by a quantity that is proportional to 1/\sqrt{n} and depends only polynomially on the number of edges of the graph. The algorithm can be implemented with linear complexity in the number of rounds n and in the number of edges. An extension to the so-called label efficient setting is also given, in which the decision maker is informed about the weights of the edges corresponding to the chosen path at a total of m << n time instances. Another extension is shown where the decision maker competes against a time-varying path, a generalization of the problem of tracking the best expert. A version of the multi-armed bandit setting for shortest path is also discussed where the decision maker learns only the total weight of the chosen path but not the weights of the individual edges on the path. Applications to routing in packet switched networks along with simulation results are also presented.
Regret Lower Bound and Optimal Algorithm in Finite Stochastic Partial Monitoring
The University of Tokyo(东京大学)
部分监控是一个一般模型的顺序学习与有限的反馈形式化为两个球员之间的游戏。在这个游戏中,学习者选择一个动作,同时对手选择一个结果,然后学习者遭受损失,并收到反馈信号。学习者的目标是使总损失最小化。在本文中,我们研究了有限行动和随机结果的部分监控。我们推导出一个对数分布相关的遗憾下限,定义了问题的难度。受DMED算法(Honda and Takemura,2010)的启发,我们提出了PM-DMED算法,该算法最小化了分布相关的后悔。PM-DMED在数值实验中的性能明显优于最先进的算法。为了证明PM-DMED在遗憾界上的最优性,我们通过引入铰链函数(PM-DMED-Hinge)对算法进行了稍微修改。然后,我们得到了一个渐进最优的后悔PM-DMED-铰链的上界匹配的下限。
Neural Active Learning Meets the Partial Monitoring Framework
Université Laval(拉瓦尔大学);Thales Research and Technology (cortAIx)(泰勒斯研究与技术(cortAIx));Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Institut Intelligence et Données(智能与数据研究所)
Robust approachability and regret minimization in games with partial monitoring
Israel Institute of Technology (Technion)(以色列理工学院);Université Paris-Diderot(巴黎第七大学(巴黎狄德罗大学));Ecole Normale Supérieure(巴黎高等师范学院);CNRS(法国国家科学研究中心);INRIA(法国国家信息与自动化研究所);HEC Paris(巴黎高等商学院)
Instance-Dependent Regret Bounds for Nonstochastic Linear Partial Monitoring
EPFL(苏黎世联邦理工学院);Università degli Studi di Milano(米兰大学);Politecnico di Milano(米兰理工大学);ETH Zürich(苏黎世联邦理工学院);Univ. Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Inria(法国国家信息与自动化技术研究院);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔INP);LJK(实验室)
在完全监控的可逼近性中,已知有两种类型的条件对于凸集是等价的:原始条件和对偶条件。原始的形式是:一个集合C是可接近的当且仅当所有包含的半空间在一次博弈中是可接近的;而对偶的形式是:一个凸集C是可接近的当且仅当它与所有具有某种形式的支付集相交。我们考虑部分监控博弈中的可接近性。在以前的工作(Perchet 2011; Mannor et al. 2011)中,我们提供了可逼近凸集的对偶特征;我们还展示了C是多胞形的情况下的有效策略。本文给出了凸集可部分监控逼近的基本条件。它们依赖于修改后的报酬函数,并导致可接近性策略,基于修改后的支付函数,进行类似于Blackwell(1956)的策略的预测;这是在对比与以前研究的策略,在这种情况下,主要依赖于信号结构,旨在估计以及所收到的信号的分布。我们的结果推广了Kohlberg 1975的经典结果(也可参见Mertens et al. 1994),并适用于具有任意信号结构的博弈以及任意凸集。