arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.06916cs.LGstat.ML
收藏

审查制度下土匪问题的有效维数

Effective Dimension in Bandit Problems under Censorship

AWS AI Labs(AWS人工智能实验室);MIT(麻省理工学院)

Gauthier Guinet, Saurabh Amin, Patrick Jaillet

摘要:

本文研究了删失环境中的多臂和上下文强盗问题。我们的目标是估计的性能损失,由于审查的背景下,经典算法设计的无审查环境。我们的主要贡献包括引进了广泛的审查模型和他们的分析方面的有效层面的问题-一个自然的措施,其潜在的统计复杂性和主要驱动程序的遗憾约束。特别是,有效维度使我们能够保持原始问题的一阶结构,同时将其嵌入到更大的空间中,从而自然会产生类似于未经审查的设置的结果。我们的分析涉及到椭圆势不等式的连续推广,我们相信这是独立的利益。我们还发现了一个有趣的属性下审查的决策:一个短暂的阶段,在此期间,最初的错误指定的审查是自我纠正在一个额外的成本,其次是一个固定的阶段,反映了固有的放缓学习的有效尺寸。我们的研究结果对于序列决策模型的应用是有用的,其中收到的反馈取决于战略不确定性(例如,代理遵循推荐的意愿)和/或随机不确定性(例如,信息的丢失或延迟到达)。

英文摘要:

In this paper, we study both multi-armed and contextual bandit problems in censored environments. Our goal is to estimate the performance loss due to censorship in the context of classical algorithms designed for uncensored environments. Our main contributions include the introduction of a broad class of censorship models and their analysis in terms of the effective dimension of the problem -- a natural measure of its underlying statistical complexity and main driver of the regret bound. In particular, the effective dimension allows us to maintain the structure of the original problem at first order, while embedding it in a bigger space, and thus naturally leads to results analogous to uncensored settings. Our analysis involves a continuous generalization of the Elliptical Potential Inequality, which we believe is of independent interest. We also discover an interesting property of decision-making under censorship: a transient phase during which initial misspecification of censorship is self-corrected at an extra cost, followed by a stationary phase that reflects the inherent slowdown of learning governed by the effective dimension. Our results are useful for applications of sequential decision-making models where the feedback received depends on strategic uncertainty (e.g., agents' willingness to follow a recommendation) and/or random uncertainty (e.g., loss or delay in arrival of information).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑