arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01853cs.LGstat.MEstat.ML
收藏

在A/B测试中设计时间序列实验的Transformer强化学习

Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learning

School of Mathematical Sciences(数学科学学院);Zhejiang University(浙江大学);Yunnan Key Laboratory of Statistical Modeling and Data Analysis(云南统计建模与数据分析重点实验室);Yunnan University(云南大学);School of Statistics(统计学系);East China Normal University(华东师范大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);London School of Economics and Political Science(伦敦政治经济学院)

Xiangkun Wu, Qianglin Wen, Yingying Zhang, Hongtu Zhu, Ting Li, Chengchun Shi

AI总结:

本文提出基于Transformer的强化学习方法,用于在A/B测试中设计时间序列实验,通过充分利用历史记录和直接优化均方误差,提升实验设计效果。

摘要:

A/B测试已成为现代科技公司进行政策评估的黄金标准。然而,将其应用于时间序列实验,即政策随时间顺序分配,仍然具有挑战性。现有设计存在两个局限性:(i)它们没有充分利用整个历史记录来分配治疗;(ii)它们依赖于强假设来近似目标函数(例如估计治疗效应的均方误差)以优化设计。我们首先建立了一个不可能定理,表明不考虑完整历史记录会导致次优设计,因为时间序列实验中存在动态依赖性。为同时解决这两个局限性,我们接下来提出了一种基于Transformer的强化学习(RL)方法,利用Transformer将分配条件于完整历史记录,并利用RL直接优化MSE,而无需依赖限制性假设。在合成数据、公开可用的调度模拟器和真实世界出租车共享数据集上的实证评估表明,我们的方法在各方面都优于现有设计。

英文摘要:

A/B testing has become a gold standard for modern technological companies to conduct policy evaluation. Yet, its application to time series experiments, where policies are sequentially assigned over time, remains challenging. Existing designs suffer from two limitations: (i) they do not fully leverage the entire history for treatment allocation; (ii) they rely on strong assumptions to approximate the objective function (e.g., the mean squared error of the estimated treatment effect) for optimizing the design. We first establish an impossibility theorem showing that failure to condition on the full history leads to suboptimal designs, due to the dynamic dependencies in time series experiments. To address both limitations simultaneously, we next propose a transformer reinforcement learning (RL) approach which leverages transformers to condition allocation on the entire history and employs RL to directly optimize the MSE without relying on restrictive assumptions. Empirical evaluations on synthetic data, a publicly available dispatch simulator, and a real-world ridesharing dataset demonstrate that our proposal consistently outperforms existing designs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑