arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.06592stat.MEcs.LG
收藏

TSEC:实验约束下的在线实验框架

TSEC: a framework for online experimentation under experimental constraints

Duke University(杜克大学);Uber Technologies Inc.(优步科技公司);Georgia Institute of Technology(佐治亚理工学院)

Simon Mak, Yuanshuo Zhou, Lavonne Hoang, C. F. Jeff Wu

摘要:

Thompson抽样算法是一种求解多臂强盗问题的流行算法,在从网站设计到投资组合优化的广泛应用中得到了广泛的应用。然而,在这样的应用中,选择(或臂)$N$的数量可能很大,并且做出自适应决策所需的数据需要昂贵的实验。然后,人们面临着在每个时间段内仅对$K \ll N$臂的一小部分进行实验的约束,这对传统的Thompson采样造成了问题。我们提出了一个新的汤普森采样实验约束(TSEC)的方法,它解决了这个所谓的“手臂预算约束”。TSEC利用贝叶斯交互模型与效果层次先验,不同的手臂上的奖励之间的相关性建模。然后将此拟合模型集成到Thompson采样中,以共同确定用于实验的良好臂子集,并在这些臂上分配资源。我们证明了TSEC的有效性在两个问题的手臂预算限制。第一个是模拟网站优化研究,TSEC显示出比行业基准明显的改善。第二个是投资组合优化的应用程序,以行业为基础的交易所交易基金,其中TSEC提供更一致和更大的财富积累超过标准的投资策略。

英文摘要:

Thompson sampling is a popular algorithm for solving multi-armed bandit problems, and has been applied in a wide range of applications, from website design to portfolio optimization. In such applications, however, the number of choices (or arms) $N$ can be large, and the data needed to make adaptive decisions require expensive experimentation. One is then faced with the constraint of experimenting on only a small subset of $K \ll N$ arms within each time period, which poses a problem for traditional Thompson sampling. We propose a new Thompson Sampling under Experimental Constraints (TSEC) method, which addresses this so-called "arm budget constraint". TSEC makes use of a Bayesian interaction model with effect hierarchy priors, to model correlations between rewards on different arms. This fitted model is then integrated within Thompson sampling, to jointly identify a good subset of arms for experimentation and to allocate resources over these arms. We demonstrate the effectiveness of TSEC in two problems with arm budget constraints. The first is a simulated website optimization study, where TSEC shows noticeable improvements over industry benchmarks. The second is a portfolio optimization application on industry-based exchange-traded funds, where TSEC provides more consistent and greater wealth accumulation over standard investment strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑