反事实置信上界：面向上下文赌博机的一种新乐观主义原则
Upper Counterfactual Confidence Bounds: a New Optimism Principle for Contextual Bandits
AI总结：
针对现有乐观算法难处理通用函数类与大上下文空间的问题，提出在策略空间构建置信界的UCCB原则，可高效处理相关场景并拓展至无限动作上下文赌博机。
摘要：
面对不确定性时的乐观主义原则是多臂赌博机与强化学习领域应用最广泛、最成功的理念之一。然而，现有的乐观算法（主要是UCB及其变体）往往难以应对通用函数类和大上下文空间。本文研究带有离线回归预言机的通用上下文赌博机，提出了一种设计乐观算法的简单通用原则，名为“反事实置信上界”（Upper Counterfactual Confidence Bounds，UCCB）。UCCB的核心创新在于在策略空间中构建置信界，而非像UCB那样在动作空间中构建。我们证明，这类算法在处理通用函数类和大上下文空间时具有可证明的最优性与计算高效性。此外，我们说明UCCB原则可无缝扩展到无限动作的通用上下文赌博机，为采用离线回归预言机的这类场景提供了首批解决方案。
英文摘要：
The principle of optimism in the face of uncertainty is one of the most widely used and successful ideas in multi-armed bandits and reinforcement learning. However, existing optimistic algorithms (primarily UCB and its variants) often struggle to deal with general function classes and large context spaces. In this paper, we study general contextual bandits with an offline regression oracle and propose a simple, generic principle to design optimistic algorithms, dubbed "Upper Counterfactual Confidence Bounds" (UCCB). The key innovation of UCCB is building confidence bounds in policy space, rather than in action space as is done in UCB. We demonstrate that these algorithms are provably optimal and computationally efficient in handling general function classes and large context spaces. Furthermore, we illustrate that the UCCB principle can be seamlessly extended to infinite-action general contextual bandits, provide the first solutions to these settings when employing an offline regression oracle.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
