arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10282cs.LGstat.ML
收藏

具有中介反馈的Bandit问题的信息容量遗憾界

Information Capacity Regret Bounds for Bandits with Mediator Feedback

Università degli Studi di Milano(米兰大学);Politecnico di Milano(米兰理工大学)

Khaled Eldowa, Nicolò Cesa-Bianchi, Alberto Maria Metelli, Marcello Restelli

AI总结:

研究中介反馈bandit问题,引入策略集容量作为复杂度度量,采用EXP4算法在对抗和随机设定下给出新遗憾界,并证明匹配下界及全信息变体的遗憾界。

摘要:

本工作研究中介反馈问题,这是一种bandit博弈,其决策集由多个策略组成,每个策略与一个在公共结果空间上的概率分布相关联。在选择一个策略后,学习者观察到从其分布中采样的结果,并承担当前轮次分配给该结果的损失。我们引入策略集容量作为策略集复杂度的信息论度量。采用经典的EXP4算法,我们在对抗性和随机性两种设定下,提供了依赖于策略集容量的新遗憾界。对于一系列策略集族,我们证明了几乎匹配的下界,其随容量的缩放方式相似。我们还考虑了策略分布在各轮次间可以变化的情况,从而解决了相关的带有专家建议的bandit问题,并改进了其先前结果。此外,我们证明了一个下界,表明在线性bandit反馈下,通常无法利用策略间的相似性。最后,对于一种全信息变体,我们提供了一个随策略集信息半径缩放的遗憾界。

英文摘要:

This work addresses the mediator feedback problem, a bandit game where the decision set consists of a number of policies, each associated with a probability distribution over a common space of outcomes. Upon choosing a policy, the learner observes an outcome sampled from its distribution and incurs the loss assigned to this outcome in the present round. We introduce the policy set capacity as an information-theoretic measure for the complexity of the policy set. Adopting the classical EXP4 algorithm, we provide new regret bounds depending on the policy set capacity in both the adversarial and the stochastic settings. For a selection of policy set families, we prove nearly-matching lower bounds, scaling similarly with the capacity. We also consider the case when the policies' distributions can vary between rounds, thus addressing the related bandits with expert advice problem, which we improve upon its prior results. Additionally, we prove a lower bound showing that exploiting the similarity between the policies is not possible in general under linear bandit feedback. Finally, for a full-information variant, we provide a regret bound scaling with the information radius of the policy set.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑