We apply the method of defensive forecasting, based on the use of game-theoretic supermartingales, to prediction with expert advice. In the traditional setting of a countable number of experts and a finite number of outcomes, the Defensive Forecasting Algorithm is very close to the well-known Aggregating Algorithm. Not only the performance guarantees but also the predictions are the same for these two methods of fundamentally different nature. We discuss also a new setting where the experts can give advice conditional on the learner's future decision. Both the algorithms can be adapted to the new setting and give the same performance guarantees as in the traditional setting. Finally, we outline an application of defensive forecasting to a setting with several loss functions.
We provide a new method for online learning, specifically prediction with expert advice, in a changing environment. In a non-changing environment the Squint algorithm has been designed to always function at least as well as other known algorithms and in specific cases it functions much better. However, when using a conventional black-box algorithm to make Squint suitable for a changing environment, it loses its beneficial properties. Hence, we provide a new algorithm, Squint-CE, which is suitable for a changing environment and preserves the properties of Squint.
We introduce a family of interpretable machine learning models, with two broad additions: Linearised Additive Models (LAMs) which replace the ubiquitous logistic link function in General Additive Models (GAMs); and SubscaleHedge, an expert advice algorithm for combining base models trained on subsets of features called subscales. LAMs can augment any additive binary classification model equipped with a sigmoid link function. Moreover, they afford direct global and local attributions of additive components to the model output in probability space. We argue that LAMs and SubscaleHedge improve th…
We study the prediction with expert advice setting, where the aim is to produce a decision by combining the decisions generated by a set of experts, e.g., independently running algorithms. We achieve the min-max optimal dynamic regret under the prediction with expert advice setting, i.e., we can compete against time-varying (not necessarily fixed) combinations of expert decisions in an optimal manner. Our end-algorithm is truly online with no prior information, such as the time horizon or loss ranges, which are commonly used by different algorithms in the literature. Both our regret guarantees…
Università degli Studi di Milano（米兰大学）；Politecnico di Milano（米兰理工大学）
We investigate the problem of bandits with expert advice when the experts are fixed and known distributions over the actions. Improving on previous analyses, we show that the regret in this setting is controlled by information-theoretic quantities that measure the similarity between experts. In some natural special cases, this allows us to obtain the first regret bound for EXP4 that can get arbitrarily close to zero if the experts are similar enough. While for a different algorithm, we provide another bound that describes the similarity between the experts in terms of the KL-divergence, and we…
Consistent Learning-to-Defer with Expert-Conditional Advice
School of Computing（计算学院）；Département de Mathématiques（数学系）；National University of Singapore（新加坡国立大学）；Sorbonne University（索邦大学）；Fédération ENAC（ENAC联合会）；ISAE-SUPAERO；ONERA；Agency for Science, Technology and Research（科技研究局）；Université de Toulouse（图卢兹大学）；Institute for Infocomm Research（信息与通信研究所）
In this paper, we study a variant of the framework of online learning using expert advice with limited/bandit feedback. We consider each expert as a learning entity, seeking to more accurately reflecting certain real-world applications. In our setting, the feedback at any time $t$ is limited in a sense that it is only available to the expert $i^t$ that has been selected by the central algorithm (forecaster), \emph{i.e.}, only the expert $i^t$ receives feedback from the environment and gets to learn at time $t$. We consider a generic black-box approach whereby the forecaster does not control or…
In this work, we aim to create a completely online algorithmic framework for prediction with expert advice that is translation-free and scale-free of the expert losses. Our goal is to create a generalized algorithm that is suitable for use in a wide variety of applications. For this purpose, we study the expected regret of our algorithm against a generic competition class in the sequential prediction by expert advice problem, where the expected regret measures the difference between the losses of our prediction algorithm and the losses of the 'best' expert selection strategy in the competition…
Tel Aviv University（特拉维夫大学）；Ben-Gurion University（本-古里安大学）；Google Research（谷歌研究院）
We revisit the fundamental problem of prediction with expert advice, in a setting where the environment is benign and generates losses stochastically, but the feedback observed by the learner is subject to a moderate adversarial corruption. We prove that a variant of the classical Multiplicative Weights algorithm with decreasing step sizes achieves constant regret in this setting and performs optimally in a wide range of environments, regardless of the magnitude of the injected corruption. Our results reveal a surprising disparity between the often comparable Follow the Regularized Leader (FTR…
Improved Regret Bounds for Bandits with Expert Advice
Università degli Studi di Milano（米兰大学）；Politecnico di Milano（米兰理工大学）；Politecnico di Torino（都灵理工大学）；Istituto Italiano di Tecnologia（意大利技术研究院）；TU Delft（代尔夫特理工大学）
Optimal Prediction Using Expert Advice and Randomized Littlestone Dimension
The Henry and Marilyn Taub Faculty of Computer Science, Technion, Israel（计算机科学学院，技术ion）；Faculty of Mathematics, Technion, Israel（数学学院，技术ion）；Department of Computer Science, Purdue University, USA（计算机科学系，普渡大学）；Google Research, Israel（谷歌研究，以色列）
IBM Research（IBM研究）；Argonne National Laboratory（阿贡国家实验室）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
We consider the problem of black-box function optimization over the boolean hypercube. Despite the vast literature on black-box function optimization over continuous domains, not much attention has been paid to learning models for optimization over combinatorial domains until recently. However, the computational complexity of the recently devised algorithms are prohibitive even for moderate numbers of variables; drawing one sample using the existing algorithms is more expensive than a function evaluation for many black-box functions of interest. To address this problem, we propose a computatio…
University of Minnesota（明尼苏达大学）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）
这项工作解决了一个经典的问题，在线预测专家的意见。我们假设一个敌对的对手，我们考虑有限的地平线和随机停止版本的零和，两人对策。针对一个适当的连续极限，利用最优控制的方法，将博弈的价值刻画为某个非线性偏微分方程的黏性解.分析还揭示了预测者和对手的最小最大最优策略。特别是，我们的工作提供了对Gravin，Peres和Sivan最近工作的连续统一观点（Proc SODA 2016）。我们的技术类似于Kohn和Serfaty（Comm Pure Appl Math 2010）的技术，其中一些两人对策的缩放限制导致椭圆或抛物偏微分方程。
Solving portfolio management problems using deep reinforcement learning has been getting much attention in finance for a few years. We have proposed a new method using experts signals and historical price data to feed into our reinforcement learning framework. Although experts signals have been used in previous works in the field of finance, as far as we know, it is the first time this method, in tandem with deep RL, is used to solve the financial portfolio management problem. Our proposed framework consists of a convolutional network for aggregating signals, another convolutional network for …
Center for Data Science, New York University（纽约大学数据科学中心）；Courant Institute of Mathematical Sciences, New York University（纽约大学科朗数学科学研究所）
This work addresses the classic machine learning problem of online prediction with expert advice. We consider the finite-horizon version of this zero-sum, two-person game. Using verification arguments from optimal control theory, we view the task of finding better lower and upper bounds on the value of the game (regret) as the problem of finding better sub- and supersolutions of certain partial differential equations (PDEs). These sub- and supersolutions serve as the potentials for player and adversary strategies, which lead to the corresponding bounds. To get explicit bounds, we use closed-fo…
University of British Columbia（不列颠哥伦比亚大学）；Google（谷歌）
Prediction with experts' advice is one of the most fundamental problems in online learning and captures many of its technical challenges. A recent line of work has looked at online learning through the lens of differential equations and continuous-time analysis. This viewpoint has yielded optimal results for several problems in online learning. In this paper, we employ continuous-time stochastic calculus in order to study the discrete-time experts' problem. We use these tools to design a continuous-time, parameter-free algorithm with improved guarantees for the quantile regret. We then develop…
Center for Data Science, New York University（纽约大学数据科学中心）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）
This work addresses the classic machine learning problem of online prediction with expert advice. A new potential-based framework for the fixed horizon version of this problem has been recently developed using verification arguments from optimal control theory. This paper extends this framework to the random (geometric) stopping version. To obtain explicit bounds, we construct potentials for the geometric version from potentials used for the fixed horizon version of the problem. This construction leads to new explicit lower and upper bounds associated with specific adversary and player strateg…
Prediction with Expert Advice under Local Differential Privacy
Department of Computer Sciences University of Wisconsin — Madison（计算机科学系威斯康星大学麦迪逊分校）；Department of Electrical and Computer Engineering University of Wisconsin — Madison（电气与计算机工程系威斯康星大学麦迪逊分校）
University of Minnesota（明尼苏达大学）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）
The prediction of a binary sequence is a classic example of online machine learning. We like to call it the 'stock prediction problem,' viewing the sequence as the price history of a stock that goes up or down one unit at each time step. In this problem, an investor has access to the predictions of two or more 'experts,' and strives to minimize her final-time regret with respect to the best-performing expert. Probability plays no role; rather, the market is assumed to be adversarial. We consider the case when there are two history-dependent experts, whose predictions are determined by the d mo…
We address the problem of sequential prediction with expert advice in a non-stationary environment with long-term memory guarantees in the sense of Bousquet and Warmuth [4]. We give a linear-time algorithm that improves on the best known regret bounds [26]. This algorithm incorporates a relative entropy projection step. This projection is advantageous over previous weight-sharing approaches in that weight updates may come with implicit costs as in for example portfolio optimization. We give an algorithm to compute this projection step in linear time, which may be of independent interest.
Princeton University（普林斯顿大学）；Microsoft Research（微软研究院）；Technion—Israel Institute of Technology（以色列理工学院）；The Hebrew University of Jerusalem（希伯来大学）；Tel Aviv University（特拉维夫大学）
我们研究的问题，预测与专家的意见时，专家的数量可能是非常大的问题，甚至是无限的。我们设计了一个算法，获得了紧遗憾界的$\widetilde{O}(εT + N + \sqrt{NT})$，其中$N$是经验$ε$覆盖数的序列的损失函数的环境。此外，我们提出了一个对冲程序，使我们能够找到最佳的$ε$在事后。最后，我们讨论了我们的算法的一些有趣的应用。我们展示了我们的算法如何适用于Hazan et al.（2016）的近似低秩专家模型，并讨论了具有有界变化的专家的情况，其中在统计和在线设置中获得的遗憾界限之间存在令人惊讶的巨大差距。
Online learning with expert advice is widely used in various machine learning tasks. It considers the problem where a learner chooses one from a set of experts to take advice and make a decision. In many learning problems, experts may be related, henceforth the learner can observe the losses associated with a subset of experts that are related to the chosen one. In this context, the relationship among experts can be captured by a feedback graph, which can be used to assist the learner's decision making. However, in practice, the nominal feedback graph often entails uncertainties, which renders…
