arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.01453cs.AI
收藏

学习推断在线顺序广告的用户隐藏状态

Learning to Infer User Hidden States for Online Sequential Advertising

Alibaba Group(阿里巴巴集团);University of Southern California(南加州大学);University College London(伦敦大学学院);Shanghai Jiao Tong University(上海交通大学);Univ. of Chinese Academy of Sciences(中国科学院大学)

Zhaoqing Peng, Junqi Jin, Lan Luo, Yaodong Yang, Rui Luo, Jun Wang, Weinan Zhang, Haiyang Xu, Miao Xu, Chuan Yu, Tiejian Luo, Han Li, Jian Xu, Kun Gai

摘要:

为了在网络广告中推动购买,优化顺序广告策略是广告主的极大兴趣,其性能和可解释性都是重要的。现有的深度强化学习方法缺乏可解释性,使得理解、诊断和进一步优化策略变得不容易。在本文中,我们提出了我们的深度意图顺序广告(DISA)方法来解决这些问题。可解释性的关键部分是理解消费者的购买意图,然而,这是不可观察的(称为隐藏状态)。在本文中,我们将这种意图建模为一个潜在变量,并将问题表述为部分可观察马尔可夫决策过程(POMDP),其中基于可观察行为推断出潜在意图。大规模的工业离线和在线实验证明了我们的方法的优越性能超过几个基线。对推导出的隐态进行了分析,结果证明了我们推导的合理性。

英文摘要:

To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method's superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑