We provide consistent random algorithms for sequential decision under partial monitoring, i.e. when the decision maker does not observe the outcomes but receives instead random feedback signals. Those algorithms have no internal regret in the sense that, on the set of stages where the decision maker chose his action according to a given law, the average payoff could not have been improved in average by using any other fixed law.
They are based on a generalization of calibration, no longer defined in terms of a Voronoi diagram but instead of a Laguerre diagram (a more general concept). This allows us to bound, for the first time in this general framework, the expected average internal -- as well as the usual external -- regret at stage $n$ by $O(n^{-1/3})$, which is known to be optimal.
We show that a version of the generalised information ratio of Lattimore and Gyorgy (2020) determines the asymptotic minimax regret for all finite-action partial monitoring games provided that (a) the standard definition of regret is used but the latent space where the adversary plays is potentially infinite; or (b) the regret introduced by Rustichini (1999) is used and the latent space is finite. Our results are complemented by a number of examples. For any $p \in [1/2,1]$ there exists an infinite partial monitoring game for which the minimax regret over $n$ rounds is $n^p$ up to subpolynomia…
University of Michigan（密歇根大学）；Florida State University（佛罗里达州立大学）；University of Vienna（维也纳大学）
In this paper, we study a learning problem in which a forecaster only observes partial information. By properly rescaling the problem, we heuristically derive a limiting PDE on Wasserstein space which characterizes the asymptotic behavior of the regret of the forecaster. Using a verification type argument, we show that the problem of obtaining regret bounds and efficient algorithms can be tackled by finding appropriate smooth sub/supersolutions of this parabolic PDE.
Robust approachability and regret minimization in games with partial monitoring
Israel Institute of Technology (Technion)（以色列理工学院）；Université Paris-Diderot（巴黎第七大学（巴黎狄德罗大学））；Ecole Normale Supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）；HEC Paris（巴黎高等商学院）
Regret Lower Bound and Optimal Algorithm in Finite Stochastic Partial Monitoring
The University of Tokyo（东京大学）
部分监控是一个一般模型的顺序学习与有限的反馈形式化为两个球员之间的游戏。在这个游戏中，学习者选择一个动作，同时对手选择一个结果，然后学习者遭受损失，并收到反馈信号。学习者的目标是使总损失最小化。在本文中，我们研究了有限行动和随机结果的部分监控。我们推导出一个对数分布相关的遗憾下限，定义了问题的难度。受DMED算法（Honda and Takemura，2010）的启发，我们提出了PM-DMED算法，该算法最小化了分布相关的后悔。PM-DMED在数值实验中的性能明显优于最先进的算法。为了证明PM-DMED在遗憾界上的最优性，我们通过引入铰链函数（PM-DMED-Hinge）对算法进行了稍微修改。然后，我们得到了一个渐进最优的后悔PM-DMED-铰链的上界匹配的下限。
Testable and Actionable Calibration for Full Swap Regret
Department of Computer Science, Aarhus University（阿arhus大学计算机科学系）；Khoury College of Computer Sciences, Northeastern University（东北大学计算机科学学院）；Northeastern University（东北大学）
Instance-Dependent Regret Bounds for Nonstochastic Linear Partial Monitoring
EPFL（苏黎世联邦理工学院）；Università degli Studi di Milano（米兰大学）；Politecnico di Milano（米兰理工大学）；ETH Zürich（苏黎世联邦理工学院）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化技术研究院）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔INP）；LJK（实验室）
The University of Tokyo（东京大学）；RIKEN AIP（理化学研究所先进智能项目中心）
We investigate finite stochastic partial monitoring, which is a general model for sequential learning with limited feedback. While Thompson sampling is one of the most promising algorithms on a variety of online decision-making problems, its properties for stochastic partial monitoring have not been theoretically investigated, and the existing algorithm relies on a heuristic approximation of the posterior distribution. To mitigate these problems, we present a novel Thompson-sampling-based algorithm, which enables us to exactly sample the target parameter from the posterior distribution. Beside…
This study considers the partial monitoring problem with $k$-actions and $d$-outcomes and provides the first best-of-both-worlds algorithms, whose regrets are favorably bounded both in the stochastic and adversarial regimes. In particular, we show that for non-degenerate locally observable games, the regret is $O(m^2 k^4 \log(T) \log(k_Π T) / Δ_{\min})$ in the stochastic regime and $O(m k^{2/3} \sqrt{T \log(T) \log k_Π})$ in the adversarial regime, where $T$ is the number of rounds, $m$ is the maximum number of distinct observations per action, $Δ_{\min}$ is the minimum suboptimality gap, and …
The significance of air pollution and the problems associated with it are fueling deployments of air quality monitoring stations worldwide. The most common approach for air quality monitoring is to rely on environmental monitoring stations, which unfortunately are very expensive both to acquire and to maintain. Hence environmental monitoring stations are typically sparsely deployed, resulting in limited spatial resolution for measurements. Recently, low-cost air quality sensors have emerged as an alternative that can improve the granularity of monitoring. The use of low-cost air quality sensor…
Bayesian optimization (BO) is a popular method for black-box optimization, which relies on uncertainty as part of its decision-making process when deciding which experiment to perform next. However, not much work has addressed the effect of uncertainty on the performance of the BO algorithm and to what extent calibrated uncertainties improve the ability to find the global optimum. In this work, we provide an extensive study of the relationship between the BO performance (regret) and uncertainty calibration for popular surrogate models and compare them across both synthetic and real-world exper…
Beijing Jiaotong University（北京交通大学）；Michigan State University（密歇根州立大学）
The advances of sensor technology enable people to monitor air quality through widely distributed low-cost sensors. However, measurements from these sensors usually encounter high biases and require a calibration step to reach an acceptable performance in down-streaming analytical tasks. Most existing calibration methods calibrate one type of sensor at a time, which we call single-task calibration. Despite the popularity of this single-task schema, it may neglect interactions among calibration tasks of different sensors, which encompass underlying information to promote calibration performance…
We study the problem of expert advice under partial bandit feedback setting and create a sequential minimax optimal algorithm. Our algorithm works with a more general partial monitoring setting, where, in contrast to the classical bandit feedback, the losses can be revealed in an adversarial manner. Our algorithm adopts a universal prediction perspective, whose performance is analyzed with regret against a general expert selection sequence. The regret we study is against a general competition class that covers many settings (such as the switching or contextual experts settings) and the expert …
We consider systems that require timely monitoring of sources over a communication network, where the cost of delayed information is unknown, time-varying and possibly adversarial. For the single source monitoring problem, we design algorithms that achieve sublinear regret compared to the best fixed policy in hindsight. For the multiple source scheduling problem, we design a new online learning algorithm called Follow-the-Perturbed-Whittle-Leader and show that it has low regret compared to the best fixed scheduling policy in hindsight, while remaining computationally feasible. The algorithm an…
Cuffless, calibration-free hemodynamic monitoring with physics-informed machine learning models
University of Utah（犹他大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Utah Health（犹他大学健康中心）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Veterans Affairs Salt Lake City Health Care System（退伍军人事务部盐湖城医疗系统）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；University of Pittsburgh（匹兹堡大学）
The climatic challenges are rising across the globe in general and in worst hit under-developed countries in particular. The need for accurate measurements and forecasting of pollutants with low-cost deployment is more pertinent today than ever before. Low-cost air quality monitoring sensors are prone to erroneous measurements, frequent downtimes, and uncertain operational conditions. Such a situation demands a prudent approach to ensure an effective and flexible calibration scheme. We propose MAQ-CaF, a modular air quality calibration, and forecasting methodology, that side-steps the challeng…
ML-based predictions are used to inform consequential decisions about individuals. How should we use predictions (e.g., risk of heart attack) to inform downstream binary classification decisions (e.g., undergoing a medical procedure)? When the risk estimates are perfectly calibrated, the answer is well understood: a classification problem's cost structure induces an optimal treatment threshold $j^{\star}$. In practice, however, some amount of miscalibration is unavoidable, raising a fundamental question: how should one use potentially miscalibrated predictions to inform binary decisions? We fo…
Finite and Corruption-Robust Regret Bounds in Online Inverse Linear Optimization under M-Convex Action Sets
Institute for Chemical Reaction Design and Discovery (ICReDD), Hokkaido University（化学反应设计与发现研究所（ICReDD），北海道大学）；D3 Center, The University of Osaka（大阪大学D3中心）；Center for Advanced Intelligence Project, RIKEN（先进智能项目中心，RIKEN）；CyberAgent, Tokyo, Japan（CyberAgent，日本东京）；National Institute of Informatics, Tokyo, Japan（信息技术国家研究所，日本东京）
Change Point--Aware Evaluation and Re-Calibration of PPG-Based Blood Pressure Estimation
VUNO Inc.（VUNO公司）；University of California, Irvine（加州大学欧文分校）；Research Institute for Future Medicine, Samsung Medical Center（三星未来医学研究所）；Inha University Hospital（仁荷大学医院）
