蒸馏汤普森采样：通过模仿学习实现实用且高效的汤普森采样
Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
AI总结：
提出基于模仿学习的蒸馏汤普森采样算法，将TS策略蒸馏为显式策略，实现低延迟决策并成功部署于Meta视频系统，显著提升视频质量与观看时间。
摘要：
汤普森采样（TS）已成为上下文老虎机问题的稳健技术。然而，TS需要后验推断和优化来生成动作，这阻碍了其在许多关注延迟和部署便捷性的在线平台上的使用。我们通过提出一种新颖的基于模仿学习的算法来使TS可操作化，该算法将TS策略蒸馏为显式策略表示，从而在移动和服务器环境中实现快速决策和轻松部署。利用在模仿策略下收集的批量数据，我们的算法迭代地对TS策略进行离线更新，并学习新的显式策略表示来模仿它。实验上，我们的模仿策略在实现超过一个数量级的决策时间延迟降低的同时，达到了与批量TS相当的性能。凭借低延迟和实现简单性，我们的算法已成功部署在Meta的多个视频上传系统中。通过随机对照试验，我们表明我们的算法在视频质量和观看时间方面带来了显著改善。
英文摘要：
Thompson sampling (TS) has emerged as a robust technique for contextual bandit problems. However, TS requires posterior inference and optimization for action generation, prohibiting its use in many online platforms where latency and ease of deployment are of concern. We operationalize TS by proposing a novel imitation-learning-based algorithm that distills a TS policy into an explicit policy representation, allowing fast decision-making and easy deployment in mobile and server-based environments. Using batched data collected under the imitation policy, our algorithm iteratively performs offline updates to the TS policy, and learns a new explicit policy representation to imitate it. Empirically, our imitation policy achieves performance comparable to batch TS while allowing more than an order of magnitude reduction in decision-time latency. Buoyed by low latency and simplicity of implementation, our algorithm has been successfully deployed in multiple video upload systems for Meta. Using a randomized controlled trial, we show our algorithm resulted in significant improvements in video quality and watch time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
