RELEAF：一种学习并利用相关性的算法
RELEAF: An Algorithm for Learning and Exploiting Relevance
AI总结：
针对高维上下文下实时决策的维度灾难问题，提出RELEAF算法，在上下文多臂老虎机框架中学习各行动的相关维度，证明其遗憾界仅依赖相关维度最大数，显著优于无相关性假设的算法。
摘要：
推荐系统、医疗诊断、网络安全等领域都需要持续的实时学习与决策。这些场景以及众多其他场景，都是大数据带来的机遇与挑战的典型代表：可用信息通常来自多种来源、具备多样特征，因此从所有来源中学习可能具有价值，但整合所学内容会受到维度灾难的制约。本文开发并分析了能够在避免维度灾难的同时实现高效学习与决策的算法。我们将学习者/决策者在特定时刻可获取的信息形式化为上下文向量，学习者在采取行动时需要考虑该向量。通常情况下上下文向量维度极高，但在很多场景中，最相关的信息仅嵌入在少数几个相关维度中。如果能提前知晓这些相关维度，问题就会很简单——但事实并非如此。此外，不同行动对应的相关维度可能有所不同。我们的算法能够学习每种行动的相关维度，并基于所学内容做出决策。形式上，我们在上下文多臂老虎机的结构基础上，新增并利用了一种相关性关系。我们证明了该算法的通用遗憾界，其时间阶仅取决于所有行动中相关维度的最大数量；在相关性关系为单值（即函数）的特殊情况下，遗憾界降至$\tilde{O}(T^{2(\u221a2-1)})$；而在不存在相关性关系的情况下，目前已知最优的上下文老虎机算法能达到的遗憾界为$\tilde{O}(T^{(D+1)/(D+2)})$，其中$D$是上下文向量的完整维度。
英文摘要：
Recommender systems, medical diagnosis, network security, etc., require on-going learning and decision-making in real time. These -- and many others -- represent perfect examples of the opportunities and difficulties presented by Big Data: the available information often arrives from a variety of sources and has diverse features so that learning from all the sources may be valuable but integrating what is learned is subject to the curse of dimensionality. This paper develops and analyzes algorithms that allow efficient learning and decision-making while avoiding the curse of dimensionality. We formalize the information available to the learner/decision-maker at a particular time as a context vector which the learner should consider when taking actions. In general the context vector is very high dimensional, but in many settings, the most relevant information is embedded into only a few relevant dimensions. If these relevant dimensions were known in advance, the problem would be simple -- but they are not. Moreover, the relevant dimensions may be different for different actions. Our algorithm learns the relevant dimensions for each action, and makes decisions based in what it has learned. Formally, we build on the structure of a contextual multi-armed bandit by adding and exploiting a relevance relation. We prove a general regret bound for our algorithm whose time order depends only on the maximum number of relevant dimensions among all the actions, which in the special case where the relevance relation is single-valued (a function), reduces to $\tilde{O}(T^{2(\sqrt{2}-1)})$; in the absence of a relevance relation, the best known contextual bandit algorithms achieve regret $\tilde{O}(T^{(D+1)/(D+2)})$, where $D$ is the full dimension of the context vector.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
