基于设计的多臂强盗推理
Design-Based Inference for Multi-arm Bandits
摘要：
多臂强盗越来越受欢迎，因为它们可以在包括临床试验、推荐系统和在线决策在内的应用领域实现真实世界的顺序决策。因此，越来越希望使用可用的自适应收集的数据集来区分一个臂是否比另一个臂更有效，例如，哪种产品或治疗更有效。不幸的是，当数据被自适应地收集或需要许多不可测试的和技术的假设时，现有的工具不能提供有效的推断，例如，平稳性，iid奖励，有界随机变量，等等。我们的论文介绍了基于设计的方法来推断多臂土匪，在那里我们条件的全部潜在的结果，并执行推断所获得的样本。我们的论文构造了有效的置信区间的奖励平均值的任何武器和任何武器之间的平均奖励差异在一个简单的方式，允许奖励是任意分布，非iid，和从非平稳分布。除了置信区间，我们还提供了有效的基于设计的置信序列，序列的置信区间具有统一的类型1错误保证随着时间的推移。置信序列允许代理在数据顺序到达时执行假设检验，并且一旦代理对推断满意就停止实验，例如，臂的平均回报在统计上显著高于期望阈值。
英文摘要：
Multi-arm bandits are gaining popularity as they enable real-world sequential decision-making across application areas, including clinical trials, recommender systems, and online decision-making. Consequently, there is an increased desire to use the available adaptively collected datasets to distinguish whether one arm was more effective than the other, e.g., which product or treatment was more effective. Unfortunately, existing tools fail to provide valid inference when data is collected adaptively or require many untestable and technical assumptions, e.g., stationarity, iid rewards, bounded random variables, etc. Our paper introduces the design-based approach to inference for multi-arm bandits, where we condition the full set of potential outcomes and perform inference on the obtained sample. Our paper constructs valid confidence intervals for both the reward mean of any arm and the mean reward difference between any arms in an assumption-light manner, allowing the rewards to be arbitrarily distributed, non-iid, and from non-stationary distributions. In addition to confidence intervals, we also provide valid design-based confidence sequences, sequences of confidence intervals that have uniform type-1 error guarantees over time. Confidence sequences allow the agent to perform a hypothesis test as the data arrives sequentially and stop the experiment as soon as the agent is satisfied with the inference, e.g., the mean reward of an arm is statistically significantly higher than a desired threshold.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
