arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.09429cs.LGstat.ML
收藏

情境线性Bandits中表示学习的复杂性

On the Complexity of Representation Learning in Contextual Linear Bandits

Meta AI

Andrea Tirinzoni, Matteo Pirotta, Alessandro Lazaric

摘要:

在上下文线性强盗中,奖励函数被假设为未知奖励向量和上下文臂对的给定嵌入的线性组合。在实践中,嵌入通常与奖励向量同时学习,从而导致在线表示学习问题。上下文强盗中的表示学习的现有方法要么是非常通用的(例如,用于学习任意函数类的模型选择技术或算法)或专用于特定结构(例如,具有某些光谱特性的嵌套特征或表示)。因此,对上下文线性强盗中表示学习代价的理解仍然是有限的。在本文中,我们采取了系统的方法来解决这个问题,并提供了一个全面的研究,通过实例依赖的角度来看。我们证明了表示学习从根本上比线性强盗更复杂(即,用给定的表示学习)。特别是,用给定的一组表示学习永远不会比用集合中最差的可实现表示学习更简单,而我们展示了它可能更难的情况。我们补充这一结果与广泛的讨论,它是如何与现有的文献,我们说明了积极的情况下,表示学习是复杂的学习与一个固定的表示和子对数遗憾是可以实现的。

英文摘要:

In contextual linear bandits, the reward function is assumed to be a linear combination of an unknown reward vector and a given embedding of context-arm pairs. In practice, the embedding is often learned at the same time as the reward vector, thus leading to an online representation learning problem. Existing approaches to representation learning in contextual bandits are either very generic (e.g., model-selection techniques or algorithms for learning with arbitrary function classes) or specialized to particular structures (e.g., nested features or representations with certain spectral properties). As a result, the understanding of the cost of representation learning in contextual linear bandit is still limited. In this paper, we take a systematic approach to the problem and provide a comprehensive study through an instance-dependent perspective. We show that representation learning is fundamentally more complex than linear bandits (i.e., learning with a given representation). In particular, learning with a given set of representations is never simpler than learning with the worst realizable representation in the set, while we show cases where it can be arbitrarily harder. We complement this result with an extensive discussion of how it relates to existing literature and we illustrate positive instances where representation learning is as complex as learning with a fixed representation and where sub-logarithmic regret is achievable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑