arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.09855math.STcs.LGstat.TH

弱信号渐近下的序贯随机实验

Weak Signal Asymptotics for Sequentially Randomized Experiments

Stanford Graduate School of Business（斯坦福大学商学院）

Xu Kuang, Stefan Wager

备注：
Forthcoming in Management Science. An earlier draft of this paper was circulated under the title "Diffusion Asymptotics for Sequential Experiments.'' Xu Kuang published under a different full name in earlier versions of this manuscript. Please use X. Kuang and S. Wager when citing this paper
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究通过弱信号渐近分析序贯随机实验，证明其收敛于扩散极限，并揭示Lipschitz连续随机化导致次优遗憾，而特定汤普森采样变体实现近最优遗憾但后验不稳定。

摘要：

我们利用弱信号渐近的视角来研究一类序贯随机实验，包括那些在解决多臂老虎机问题中出现的实验。在一个具有$n$个时间步的实验中，我们让各动作之间的平均奖励差距按$1/\sqrt{n}$的量级缩放，以便在$n$增长时保持学习任务的难度。在该机制下，我们证明了一类序贯随机实验的样本路径——这些实验适应于该缩放机制，且其臂选择概率随状态连续变化——弱收敛于一个扩散极限，该极限由随机微分方程的解给出。扩散极限使我们能够推导出随机动力学的精细的、实例特定的刻画，并获得了关于多个序贯实验（包括汤普森采样，但不包括UCB，因为UCB不满足我们的连续性假设）的遗憾和信念演化的若干见解。我们表明，所有随机化概率对观测数据具有Lipschitz连续依赖的序贯实验，在奖励差距相对较大时会遭受次优的遗憾性能。相反，我们发现，具有渐近无信息先验方差的汤普森采样版本实现了接近最优的实例特定遗憾缩放，即使在奖励差距较大时也是如此，但这些良好的遗憾性质是以高度不稳定的后验信念为代价的。

英文摘要：

We use the lens of weak signal asymptotics to study a class of sequentially randomized experiments, including those that arise in solving multi-armed bandit problems. In an experiment with $n$ time steps, we let the mean reward gaps between actions scale to the order $1/\sqrt{n}$ so as to preserve the difficulty of the learning task as $n$ grows. In this regime, we show that the sample paths of a class of sequentially randomized experiments -- adapted to this scaling regime and with arm selection probabilities that vary continuously with state -- converge weakly to a diffusion limit, given as the solution to a stochastic differential equation. The diffusion limit enables us to derive refined, instance-specific characterization of stochastic dynamics, and to obtain several insights on the regret and belief evolution of a number of sequential experiments including Thompson sampling (but not UCB, which does not satisfy our continuity assumption). We show that all sequential experiments whose randomization probabilities have a Lipschitz-continuous dependence on the observed data suffer from sub-optimal regret performance when the reward gaps are relatively large. Conversely, we find that a version of Thompson sampling with an asymptotically uninformative prior variance achieves near-optimal instance-specific regret scaling, including with large reward gaps, but these good regret properties come at the cost of highly unstable posterior beliefs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑