arXiv 2102.063852021-02cs.LGcs.DS

The Symmetry between Arms and Knapsacks: A Primal-Dual Approach for Bandits with Knapsacks

Imperial College Business School（帝国理工学院商学院）；Stanford University（斯坦福大学）

Xiaocheng Li, Chunlin Sun, Yinyu Ye

英文摘要：

In this paper, we study the bandits with knapsacks (BwK) problem and develop a primal-dual based algorithm that achieves a problem-dependent logarithmic regret bound. The BwK problem extends the multi-arm bandit (MAB) problem to model the resource consumption associated with playing each arm, and the existing BwK literature has been mainly focused on deriving asymptotically optimal distribution-free regret bounds. We first study the primal and dual linear programs underlying the BwK problem. From this primal-dual perspective, we discover symmetry between arms and knapsacks, and then propose a new notion of sub-optimality measure for the BwK problem. The sub-optimality measure highlights the important role of knapsacks in determining algorithm regret and inspires the design of our two-phase algorithm. In the first phase, the algorithm identifies the optimal arms and the binding knapsacks, and in the second phase, it exhausts the binding knapsacks via playing the optimal arms through an adaptive procedure. Our regret upper bound involves the proposed sub-optimality measure and it has a logarithmic dependence on length of horizon $T$ and a polynomial dependence on $m$ (the numbers of arms) and $d$ (the number of knapsacks). To the best of our knowledge, this is the first problem-dependent logarithmic regret bound for solving the general BwK problem.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

带背包的聚类线性上下文老虎机

Clustered Linear Contextual Bandits with Knapsacks

The University of Washington（华盛顿大学）；Northwestern University（西北大学）；Adobe Research（奥多比研究院）

针对聚类上下文老虎机中需同时学习奖励/消耗模型与未知聚类归属、且受资源约束的问题，提出一种仅对随机臂子集做一次聚类的算法，结合计量经济学与带约束老虎机技术实现次线性遗憾。

2023-08-21cs.LGstat.ML

带有任意时刻背包的多臂老虎机

Bandits with Anytime Knapsacks

Carnegie Mellon University（卡内基梅隆大学）；Bilkent University（比尔肯特大学）

研究带有任意时刻成本约束的多臂老虎机问题，提出SUAK算法，利用上置信界平衡探索与利用，在可能违反约束时跳过轮次，实现了O(K log T)的遗憾上界。

2026-10-12cs.LG

带可补给背包的多臂老虎机：两全其美

Bandits with Replenishable Knapsacks: the Best of both Worlds

Politecnico di Milano（米兰理工大学）；Bocconi University（博科尼大学）；Sapienza Università di Roma（罗马第一大学）

该研究提出了一种带资源补给的背包多臂老虎机原始-对偶框架，在对抗输入下获常数竞争比，随机输入下得与实例无关的Õ(T^1/2)遗憾界，并给出了经济问题应用。

2023-06-14cs.LGstat.ML

Bandits with Knapsacks

Google Research（谷歌研究院）；Cornell University（康奈尔大学）；Microsoft Research（微软研究院）

Multi-armed bandit problems are the predominant theoretical model of exploration-exploitation tradeoffs in learning, and they have countless applications ranging from medical trials, to communication networks, to Web search and advertising. In many of these application domains the learner may be constrained by one or more supply (or budget) limits, in addition to the customary limitation on the time horizon. The literature lacks a general model encompassing these sorts of problems. We introduce such a model, called "bandits with knapsacks", that combines aspects of stochastic integer programmi…

2017-09-05cs.DScs.LG

Knapsack based Optimal Policies for Budget-Limited Multi-Armed Bandits

The University of Sydney Business School（悉尼大学商学院）；University of Southampton（南安普顿大学）

In budget-limited multi-armed bandit (MAB) problems, the learner's actions are costly and constrained by a fixed budget. Consequently, an optimal exploitation policy may not be to pull the optimal arm repeatedly, as is the case in other variants of MAB, but rather to pull the sequence of different arms that maximises the agent's total reward within the budget. This difference from existing MABs means that new approaches to maximising the total reward are required. Given this, we develop two pulling policies, namely: (i) KUBE; and (ii) fractional KUBE. Whereas the former provides better perform…

2012-04-09cs.AIcs.LG

高维背包线性赌博机

High-dimensional Linear Bandits with Knapsacks

Department of Statistics and Data Science, University of Pennsylvania（统计与数据科学系，宾夕法尼亚大学）；Department of Mathematics, The Hong Kong University of Science and Technology（数学系，香港科学与技术大学）；Department of Industrial Engineering & Decision Analytics, The Hong Kong University of Science and Technology（工业工程与决策分析系，香港科学与技术大学）

针对高维线性背包赌博机问题，提出结合在线硬阈值稀疏估计与原始-对偶更新的算法，实现随特征维度对数增长的次线性遗憾，并在特定条件下达到最优遗憾界。

2026-09-09cs.DScs.LGstat.ML

更多相关论文收起更多论文

抗腐败的带背包约束的稀疏线性上下文赌博机

Corruption-Robust Sparse Linear Contextual Bandits with Knapsack Constraints

The Hong Kong University of Science and Technology（香港科技大学）；University of Pennsylvania（宾夕法尼亚大学）

针对带背包约束的稀疏线性上下文赌博机，提出抗腐败的ROPD和Shared-Grid算法，在奖励与消费腐败下实现亚线性遗憾并保持预算安全。

2026-09-30cs.LG

超越原始-对偶方法的带随机与对抗约束的赌博机问题

Beyond Primal-Dual Methods in Bandits with Stochastic and Adversarial Constraints

Bocconi university（博科尼大学）；Politecnico di Milano（米兰理工大学）；Sapienza University of Rome（罗马大学）

针对带任意长期约束的赌博机问题，提出基于乐观约束估计的两全其美算法，无需斯莱特条件，在随机与对抗约束下均达最优性能，且遗憾界对约束数量取对数。

2024-05-25cs.LG

Budgeted Combinatorial Multi-Armed Bandits

We consider a budgeted combinatorial multi-armed bandit setting where, in every round, the algorithm selects a super-arm consisting of one or more arms. The goal is to minimize the total expected regret after all rounds within a limited budget. Existing techniques in this literature either fix the budget per round or fix the number of arms pulled in each round. Our setting is more general where based on the remaining budget and remaining number of rounds, the algorithm can decide how many arms to be pulled in each round. First, we propose CBwK-Greedy-UCB algorithm, which uses a greedy techniqu…

2022-02-13cs.LGcs.MA

An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits

Politecnico di Milano（米兰理工大学）；Facebook AI Research（脸书人工智能研究院）

In the contextual linear bandit setting, algorithms built on the optimism principle fail to exploit the structure of the problem and have been shown to be asymptotically suboptimal. In this paper, we follow recent approaches of deriving asymptotically optimal algorithms from problem-dependent regret lower bounds and we introduce a novel algorithm improving over the state-of-the-art along multiple dimensions. We build on a reformulation of the lower bound, where context distribution and exploration policy are decoupled, and we obtain an algorithm robust to unbalanced context distributions. Then…

2020-11-20cs.LGstat.ML

多玩家组合半老虎机问题

Multi-Play Combinatorial Semi-Bandit Problem

University of Tokyo（东京大学）；CENTAI Institute（CENTAI研究院）；Microsoft Research（微软研究院）

针对组合半老虎机（CSB）问题在二元决策空间上的限制，提出了多玩家组合半老虎机（MP-CSB）问题，允许选择非负整数动作并观察多个反馈，并设计了两种算法分别实现 $O(\log T)$ 分布相关遗憾和两全其美遗憾保证。

2025-09-12cs.LG

无悔是不够的！通过自适应遗憾最小化求解带一般约束的老虎机问题

No-Regret is not enough! Bandits with General Constraints through Adaptive Regret Minimization

Bocconi university（博科尼大学）；Politecnico di Milano（米兰理工大学）

针对带一般长期约束的 BwK，本文利用弱自适应原始-对偶遗憾最小化器的自界性质，在随机与对抗环境下保证次线性遗憾或紧竞争比，并得到对抗上下文线性约束老虎机的首个 no-$α$-regret 结果。

2024-05-10cs.LGstat.ML

Non-stationary Bandits with Knapsacks

Imperial College London（伦敦帝国理工学院）；NYU Stern School of Business（纽约大学斯特恩商学院）

In this paper, we study the problem of bandits with knapsacks (BwK) in a non-stationary environment. The BwK problem generalizes the multi-arm bandit (MAB) problem to model the resource consumption associated with playing each arm. At each time, the decision maker/player chooses to play an arm, and s/he will receive a reward and consume certain amount of resource from each of the multiple resource types. The objective is to maximize the cumulative reward over a finite horizon subject to some knapsack constraints on the resources. Existing works study the BwK problem under either a stochastic o…

2022-10-12cs.LGstat.ML

The Online Saddle Point Problem and Online Convex Optimization with Knapsacks

We study the online saddle point problem, an online learning problem where at each iteration a pair of actions need to be chosen without knowledge of the current and future (convex-concave) payoff functions. The objective is to minimize the gap between the cumulative payoffs and the saddle point value of the aggregate payoff function, which we measure using a metric called "SP-Regret". The problem generalizes the online convex optimization framework but here we must ensure both players incur cumulative payoffs close to that of the Nash equilibrium of the sum of the games. We propose an algorit…

2020-04-05cs.LGmath.OCstat.ML

适配器增强的带状机用于在线多约束多模态推断调度

Adapter-Augmented Bandits for Online Multi-Constrained Multi-Modal Inference Scheduling

School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, 510006, China.（中山大学计算机科学与工程学院）；School of Computing, Macquarie University, NSW 2109, Australia（麦考瑞大学计算机学院）

M-CMAB通过多适配器增强框架，实现多模态任务调度中的多约束优化，提升推断效率和预算利用效率。

2026-03-06cs.LG

Stochastic Bandits for Multi-platform Budget Optimization in Online Advertising

Facebook Core Data Science（Facebook核心数据科学部门）；Sapienza University of Rome（罗马大学）；Facebook AI Applied Research（Facebook人工智能应用研究部门）

We study the problem of an online advertising system that wants to optimally spend an advertiser's given budget for a campaign across multiple platforms, without knowing the value for showing an ad to the users on those platforms. We model this challenging practical application as a Stochastic Bandits with Knapsacks problem over $T$ rounds of bidding with the set of arms given by the set of distinct bidding $m$-tuples, where $m$ is the number of platforms. We modify the algorithm proposed in Badanidiyuru \emph{et al.,} to extend it to the case of multiple platforms to obtain an algorithm for b…

2021-03-25cs.GTcs.LG