首页/论文/2006.03378

arXiv 2006.033782020-06math.STstat.MLstat.TH

Adaptation to the Range in $K$-Armed Bandits

Hédi Hadiji, Gilles Stoltz

英文摘要：

We consider stochastic bandit problems with $K$ arms, each associated with a bounded distribution supported on the range $[m,M]$. We do not assume that the range $[m,M]$ is known and show that there is a cost for learning this range. Indeed, a new trade-off between distribution-dependent and distribution-free regret bounds arises, which prevents from simultaneously achieving the typical $\ln T$ and $\sqrt{T}$ bounds. For instance, a $\sqrt{T}$}distribution-free regret bound may only be achieved if the distribution-dependent regret bounds are at least of order $\sqrt{T}$. We exhibit a strategy achieving the rates for regret indicated by the new trade-off.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A Confirmation of a Conjecture on the Feldman's Two-armed Bandit Problem

Myopic strategy is one of the most important strategies when studying bandit problems. In this paper, we consider the two-armed bandit problem proposed by Feldman. With general distributions and utility functions, we obtain a necessary and sufficient condition for the optimality of the myopic strategy. As an application, we could solve Nouiehed and Ross's conjecture for Bernoulli two-armed bandit problems that myopic strategy stochastically maximizes the number of wins.

2022-06-02math.STstat.MLstat.TH

Exponential two-armed bandit problem

We consider exponential two-armed bandit problem in which incomes are described by exponential distribution densities. We develop Bayesian approach and present recursive equation for determination of Bayesian strategy and Bayesian risk. In the limiting case as the control horizon goes to infinity, we obtain the second order partial differential equation in the domain of "close distributions". Results are compared with Gaussian two-armed bandit. It turned out that exponential and Gaussian two-armed bandits have the same description in the limiting case. Since Gaussian two-armed bandit describes…

2019-08-15math.STstat.TH

Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewards

University of Minnesota（明尼苏达大学）

We study a multi-armed bandit problem with covariates in a setting where there is a possible delay in observing the rewards. Under some mild assumptions on the probability distributions for the delays and using an appropriate randomization to select the arms, the proposed strategy is shown to be strongly consistent.

2019-09-04cs.LGmath.STstat.ML

PAC-Bayesian aggregation and multi-armed bandits

This habilitation thesis presents several contributions to (1) the PAC-Bayesian analysis of statistical learning, (2) the three aggregation problems: given d functions, how to predict as well as (i) the best of these d functions (model selection type aggregation), (ii) the best convex combination of these d functions, (iii) the best linear combination of these d functions, (3) the multi-armed bandit problems.

2010-11-15math.STstat.TH

A new approach to Poissonian two-armed bandit problem

We consider a continuous time two-armed bandit problem in which incomes are described by Poissonian processes. We develop Bayesian approach with arbitrary prior distribution. We present two versions of recursive equation for determination of Bayesian piece-wise constant strategy and Bayesian risk and partial differential equation in the limiting case. Unlike the previously considered Bayesian settings our description uses current history of the process and not evolution of the posterior distribution.

2019-07-13math.STstat.TH

Bounded regret in stochastic multi-armed bandits

Princeton University（普林斯顿大学）；Université Paris Diderot（巴黎狄德罗大学）

We study the stochastic multi-armed bandit problem when one knows the value $μ^{(\star)}$ of an optimal arm, as a well as a positive lower bound on the smallest positive gap $Δ$. We propose a new randomized policy that attains a regret {\em uniformly bounded over time} in this setting. We also prove several lower bounds, which show in particular that bounded regret is not possible if one only knows $Δ$, and bounded regret of order $1/Δ$ is not possible if one only knows $μ^{(\star)}$

2013-02-12cs.LGmath.STstat.ML

更多相关论文收起更多论文

A Finite-Time Analysis of Multi-armed Bandits Problems with Kullback-Leibler Divergences

We consider a Kullback-Leibler-based algorithm for the stochastic multi-armed bandit problem in the case of distributions with finite supports (not necessarily known beforehand), whose asymptotic regret matches the lower bound of \cite{Burnetas96}. Our contribution is to provide a finite-time analysis of this algorithm; we get bounds whose main terms are smaller than the ones of previously known algorithms with finite-time analyses (like UCB-type algorithms).

2011-05-29math.STstat.TH

Polynomial Cost of Adaptation for X -Armed Bandits

Laboratoire de Mathématiques d’Orsay（奥赛数学实验室）；Université Paris-Sud（巴黎南大学）

In the context of stochastic continuum-armed bandits, we present an algorithm that adapts to the unknown smoothness of the objective function. We exhibit and compute a polynomial cost of adaptation to the H{ö}lder regularity for regret minimization. To do this, we first reconsider the recent lower bound of Locatelli and Carpentier [20], and define and characterize admissible rate functions. Our new algorithm matches any of these minimal rate functions. We provide a finite-time analysis and a thorough discussion about asymptotic optimality.

2019-12-09cs.LGmath.STstat.ML

The Multi-Armed Bandit Problem: An Efficient Non-Parametric Solution

Lai and Robbins (1985) and Lai (1987) provided efficient parametric solutions to the multi-armed bandit problem, showing that arm allocation via upper confidence bounds (UCB) achieves minimum regret. These bounds are constructed from the Kullback-Leibler information of the reward distributions, estimated from specified parametric families. In recent years there has been renewed interest in the multi-armed bandit problem due to new applications in machine learning algorithms and data analytics. Non-parametric arm allocation procedures like $ε$-greedy, Boltzmann exploration and BESA were studied…

2019-01-16math.STstat.TH

Regret Analysis of the Finite-Horizon Gittins Index Strategy for Multi-Armed Bandits

University of Alberta（阿尔伯塔大学）

I analyse the frequentist regret of the famous Gittins index strategy for multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it turns out that this approach leads to finite-time regret guarantees comparable to those available for the popular UCB algorithm. Along the way I derive finite-time bounds on the Gittins index that are asymptotically exact and may be of independent interest. I also discuss some computational issues and present experimental results suggesting that a particular version of the Gittins index strategy is a modest improvement on existing algorithms with…

2016-05-27cs.LGmath.STstat.ML

Multi-Armed Bandit Problem and Batch UCB Rule

We obtain the upper bound of the loss function for a strategy in the multi-armed bandit problem with Gaussian distributions of incomes. Considered strategy is an asymptotic generalization of the strategy proposed by J. Bather for the multi-armed bandit problem and using UCB rule, i.e. choosing the action corresponding to the maximum of the upper bound of the confidence interval of the current estimate of the expected value of one-step income. Results are obtained with the help of invariant description of the control on the unit horizon in the domain of close distributions because just there th…

2019-02-01math.STstat.TH

The multi-armed bandit problem with covariates

Université Paris Diderot（巴黎第七大学）；Princeton University（普林斯顿大学）

We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side information is available. We adopt a nonparametric model where the expected rewards are smooth functions of the covariate and where the hardness of the problem is captured by a margin parameter. To maximize the expected cumulative reward, we introduce a policy called Adaptively Binn…

2013-05-24cs.LGmath.STstat.ML

上下文多臂老虎机的迁移学习

Transfer Learning for Contextual Multi-armed Bandits

University of Michigan（密歇根大学）；University of Pennsylvania（宾夕法尼亚大学）

针对协变量偏移下的非参数上下文多臂老虎机，本文提出达到极小极大遗憾的迁移学习算法，并给出可自适应未知平滑度的近最优数据驱动方法。

2024-01-25cs.LGmath.STstat.ML

How to gamble with non-stationary $\mathcal{X}$-armed bandits and have no regrets

WIAS Berlin（柏林魏尔斯特拉斯应用分析与随机学研究所）

In $\mathcal{X}$-armed bandit problem an agent sequentially interacts with environment which yields a reward based on the vector input the agent provides. The agent's goal is to maximise the sum of these rewards across some number of time steps. The problem and its variations have been a subject of numerous studies, suggesting sub-linear and some times optimal strategies. The given paper introduces a novel variation of the problem. We consider an environment, which can abruptly change its behaviour an unknown number of times. To that end we propose a novel strategy and prove it attains sub-lin…

2021-01-17cs.LGmath.STstat.ML

Batch Data Processing and Gaussian Two-Armed Bandit

We consider the two-armed bandit problem as applied to data processing if there are two alternative processing methods available with different a priori unknown efficiencies. One should determine the most effective method and provide its predominant application. Gaussian two-armed bandit describes the batch, and possibly parallel, processing when the same methods are applied to sufficiently large packets of data and accumulated incomes are used for the control. If the number of packets is large enough then such control does not deteriorate the control performance, i.e. does not increase the mi…

2017-04-12math.STstat.TH

Continuum-Armed Bandits: A Function Space Perspective

Carnegie Mellon University（卡内基梅隆大学）

Continuum-armed bandits (a.k.a., black-box or $0^{th}$-order optimization) involves optimizing an unknown objective function given an oracle that evaluates the function at a query point, with the goal of using as few query points as possible. In the most well-studied case, the objective function is assumed to be Lipschitz continuous and minimax rates of simple and cumulative regrets are known in both noiseless and noisy settings. This paper studies continuum-armed bandits under more general smoothness conditions, namely Besov smoothness conditions, on the objective function. In both noiseless …

2021-03-21cs.ITcs.LGmath.IT

高斯多臂老虎机UCB规则的极限描述的随机微分方程

Stochastic differential equations for limiting description of UCB rule for Gaussian multi-armed bandits

Yaroslav-the-Wise Novgorod State University（智者雅罗斯拉夫诺夫哥罗德国立大学）

本文针对高斯多臂老虎机UCB策略，用随机微分方程和常微分方程构建极限描述，并通过蒙特卡洛模拟验证，估计最小控制视界。

2023-05-11cs.LGmath.OCmath.ST