arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1902.03588cs.GTcs.AI
收藏

广告交易中代理的最佳响应策略学习

Learning Best Response Strategies for Agents in Ad Exchanges

School of Informatics, University of Edinburgh(爱丁堡大学信息学院)

Stavros Gerakaris, Subramanian Ramamoorthy

摘要:

广告交换广泛用于在线显示广告的平台。在这些交易所中运作的自主智能体必须学习策略,以便与多样化、不断变化但未知的市场进行有利可图的互动。我们认为这个问题从出版商的角度来看,战略性地与广告客户通过公布的价格机制进行互动。这个代理的学习问题由于信息被审查的事实而变得困难,即,出版商知道印象是否被销售,但不知道其他定量信息。我们解决这个问题,使用Harsanyi-Bellman特设协调(HBA)算法,它概念化这种相互作用的随机贝叶斯博弈,并达到最佳的行动,最好的反应相对于概率信念保持在一组候选对手的行为配置文件。我们适应和应用HBA的审查信息设置的广告交易所。此外,解决的情况下,随机对手,我们设计了一个战略的基础上,Kaplan-Meier估计对手建模。我们评估所提出的方法使用模拟,其中我们表明,HBA-KM实现了更好的竞争比和更低的方差比基线的回报,包括Q-学习代理和基于UCB的在线学习代理,并媲美离线最优算法。

英文摘要:

Ad exchanges are widely used in platforms for online display advertising. Autonomous agents operating in these exchanges must learn policies for interacting profitably with a diverse, continually changing, but unknown market. We consider this problem from the perspective of a publisher, strategically interacting with an advertiser through a posted price mechanism. The learning problem for this agent is made difficult by the fact that information is censored, i.e., the publisher knows if an impression is sold but no other quantitative information. We address this problem using the Harsanyi-Bellman Ad Hoc Coordination (HBA) algorithm, which conceptualises this interaction in terms of a Stochastic Bayesian Game and arrives at optimal actions by best responding with respect to probabilistic beliefs maintained over a candidate set of opponent behaviour profiles. We adapt and apply HBA to the censored information setting of ad exchanges. Also, addressing the case of stochastic opponents, we devise a strategy based on a Kaplan-Meier estimator for opponent modelling. We evaluate the proposed method using simulations wherein we show that HBA-KM achieves substantially better competitive ratio and lower variance of return than baselines, including a Q-learning agent and a UCB-based online learning agent, and comparable to the offline optimal algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑