arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.06565cs.LGcs.GT
收藏

利用截断反馈学习资源分配

Learning to Allocate Resources with Censored Feedback

FairPlay Joint Team, Inria, France(Inria法国团队);CREST, ENSAE, Institut Polytechnique de Paris(Institut Polytechnique de Paris的CREST和ENSAE);University of Illinois, Chicago (UIC)(伊利诺伊大学芝加哥分校);Criteo AI Lab, Paris, France(Criteo巴黎AI实验室)

Giovanni Montanari, Côme Fiegel, Corentin Pla, Aadirupa Saha, Vianney Perchet

AI总结:

本文提出RA-UCB和MG-UCB算法,解决具有截断反馈的在线资源分配问题,证明了信息论的下界并验证了理论结果。

摘要:

我们研究了一个在线资源分配问题,在每个回合中,必须在K个臂上分配预算B,且反馈被截断。一个臂产生奖励当且仅当两个条件得到满足:(i) 根据特定臂的伯努利随机变量激活,该变量的参数未知;(ii) 分配的预算超过从参数分布中抽取的随机阈值,该分布的参数未知。在T个回合中,学习者必须同时估计未知参数并分配预算,以最大化累积奖励,面对探索-利用权衡。我们证明了一个信息论的后悔下界Ω(T^{1/3}),展示了该问题的内在难度。然后我们提出了RA-UCB算法,该算法利用非平凡的参数估计和置信区间。当预算B在每个回合开始时已知,RA-UCB的后悔为O(√T)的量级,甚至在更强的假设下为O(poly-log T)。对于未知的、回合依赖的预算,我们引入了MG-UCB,它允许回合内切换和微小分配,并与RA-UCB的后悔保证相匹配。然后我们通过真实世界数据集的实验验证了我们的理论结果。

英文摘要:

We study an online resource allocation problem in which at each round, a budget $B$ must be allocated across $K$ arms under censored feedback. An arm yields a reward if and only if two conditions are satisfied: (i) the arm is activated according to an arm-specific Bernoulli random variable with unknown parameter, and (ii) the allocated budget exceeds a random threshold drawn from a parametric distribution with unknown parameter. Over $T$ rounds, the learner must jointly estimate the unknown parameters and allocate the budget so as to maximize cumulative reward facing the exploration--exploitation trade-off. We prove a worst-case regret lower bound $Ω(\sqrt{T})$ for the general model, even with known uniform threshold distributions. We then propose RA-UCB, an optimistic algorithm that leverages non-trivial parameter estimation and confidence bounds. When the budget $B$ is known at the beginning of each round, RA-UCB achieves a regret bound of order $\widetilde{\mathcal{O}}(\sqrt{T})$, and even $\mathcal{O}(\mathrm{poly}\text{-}\log T)$ under stronger assumptions. As for unknown, round dependent budget, we introduce \texttt{MG-UCB}, which allows within-round switching and infinitesimal allocations, and matches the regret guarantees of RA-UCB. We then validate our theoretical results through experiments on real-world datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑