AI 中文总结
针对批量多臂老虎机，提出比值比汤普森采样（OR-TS），携带对比后验并每批重估共同水平，优于绝对率记忆，在移动水平下显著降低遗憾并更频繁选中最优臂。
AI 中文摘要
批量多臂老虎机按服务自身的调度进行更新，通常的实现方式会将每个臂的绝对奖励率从一次更新传递到下一次。当共享水平在批次之间移动时，即使臂之间的对比可能并未改变，该记忆也会变得过时。比值比汤普森采样（OR-TS）转而携带对数比值比对比的联合后验，并在每个批次中重新拟合共同水平，将其边缘化。本文明确了该更新，将其置于一个具有两个控制参数的贝叶斯老虎机智能体之中：衰减（决定更新后多少过去证据得以保留）和攻击性（决定信念转化为分配的锐利程度），并与绝对率记忆进行了评估。在86个公开A/B序列中，水平的变化幅度约为对比的二十五倍。在预设的合成环境中，移动水平使绝对率记忆的遗憾增加五倍，并在20次运行中的7次中使最佳臂低于多数流量，而OR-TS则没有这种情况。在基于71个真实实验构建的策略模拟中，由于对比过小而无法分辨，其中58个实验的预期点击差异保持在0.1%以内，但对比记忆最终落在较优臂上的频率仍超过两倍。当对比本身移动时，该策略失败，此情况也已报告。
英文摘要
Batched multi-armed bandits update on a service's own schedule, and the usual implementation carries each arm's absolute reward rate from one update to the next. When the shared level moves between batches, that memory goes stale even though the comparisons between arms may not have. Odds-Ratio Thompson Sampling (OR-TS) instead carries the joint posterior over log-odds contrasts and fits the common level afresh in every batch, marginalizing it out. This paper specifies that update, places it inside a Bayesian bandit agent with two controls, decay for how much past evidence survives an update and aggressiveness for how sharply belief becomes allocation, and evaluates it against absolute-rate memory. Across 86 public A/B series the level varies about twenty-five times more than the contrast. In prespecified synthetic environments a moving level costs absolute-rate memory five times the regret and leaves the best arm below a majority of traffic in 7 of 20 runs, against none for OR-TS. In a policy simulation built from 71 real experiments, where the contrasts are too small to resolve, expected-click differences stay within 0.1% for 58 of them, yet contrast memory still ends on the better arm more than twice as often. Where the contrasts themselves move, the bet fails, and that case is reported too.
Comments25 pages, 7 figures, 2 tables in the main text; 2 figures and 3 tables in the supplement. Reference implementation at https://github.com/sulgik/orts
