arXiv 1702.048252017-02-17cs.LG

学会使用学习者的建议

Learning to Use Learners' Advice

ETH Zurich（苏黎世联邦理工学院）

Adish Singla, Hamed Hassani, Andreas Krause

摘要：

在本文中，我们研究了一个变体的框架，在线学习使用专家意见有限/土匪反馈。我们将每个专家视为一个学习实体，寻求更准确地反映某些现实世界的应用。在我们的设置中，在任何时间$t$的反馈在某种意义上是有限的，即它仅对已经由中央算法（预报器）\emph{i.e.}选择的专家$i^t$可用，只有专家$i^t$从环境接收反馈并且在时间$t$开始学习。我们认为一个通用的黑箱方法，预测不控制或知道的学习动力学的专家除了知道以下的无遗憾学习属性：平均遗憾的任何专家$j$消失的速度至少$O(t_j^{\regretRate-1})$与$t_j$学习步骤，其中$\regretRate \in [0, 1]$是一个参数。本着在多武装匪徒问题中与事后最佳行动竞争的精神，我们的目标是在w.r.t.该算法通过遵循总是选择一个专家的策略可以接收的累积损失。我们证明了以下的硬度结果：没有任何协调之间的预测和专家，这是不可能设计一个预测实现无遗憾的保证。为了规避这种困难的结果，我们考虑一个实际的假设，允许预测者“指导”的学习过程中的专家通过过滤/阻止他们观察到的反馈从环境中，\emph{i.e.}，不允许选定的专家$i^t$在时间$t$学习一些时间步长。然后，我们设计了一个新的无遗憾学习算法\algo，通过仔细指导专家观察到的反馈，这个问题设置。我们证明了\algo在$\Time$时间步长后达到了$O(\Time^\frac{1}{2 - \regretRate})$的最坏情况预期累积遗憾。

英文摘要：

In this paper, we study a variant of the framework of online learning using expert advice with limited/bandit feedback. We consider each expert as a learning entity, seeking to more accurately reflecting certain real-world applications. In our setting, the feedback at any time $t$ is limited in a sense that it is only available to the expert $i^t$ that has been selected by the central algorithm (forecaster), \emph{i.e.}, only the expert $i^t$ receives feedback from the environment and gets to learn at time $t$. We consider a generic black-box approach whereby the forecaster does not control or know the learning dynamics of the experts apart from knowing the following no-regret learning property: the average regret of any expert $j$ vanishes at a rate of at least $O(t_j^{\regretRate-1})$ with $t_j$ learning steps where $\regretRate \in [0, 1]$ is a parameter. In the spirit of competing against the best action in hindsight in multi-armed bandits problem, our goal here is to be competitive w.r.t. the cumulative losses the algorithm could receive by following the policy of always selecting one expert. We prove the following hardness result: without any coordination between the forecaster and the experts, it is impossible to design a forecaster achieving no-regret guarantees. In order to circumvent this hardness result, we consider a practical assumption allowing the forecaster to "guide" the learning process of the experts by filtering/blocking some of the feedbacks observed by them from the environment, \emph{i.e.}, not allowing the selected expert $i^t$ to learn at time $t$ for some time steps. Then, we design a novel no-regret learning algorithm \algo for this problem setting by carefully guiding the feedbacks observed by experts. We prove that \algo achieves the worst-case expected cumulative regret of $O(\Time^\frac{1}{2 - \regretRate})$ after $\Time$ time steps.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文