基于模型的有约束MDP序列激励营销预算分配
Model-based Constrained MDP for Budget Allocation in Sequential Incentive Marketing
摘要：
序贯激励营销是网络企业获取客户、提高忠诚度、促进销售的重要途径。如何有效地分配激励措施以实现回报最大化（例如，然而，在文献中较少研究预算约束下的业务目标）。这个问题在技术上是具有挑战性的，这是由于以下事实：1）必须使用历史记录的数据来学习分配策略，这在本质上是反事实的，以及2）最优性和可行性（即，该费用不能超过预算）需要在部署到在线系统之前进行评估。在本文中，我们制定了一个约束马尔可夫决策过程（CMDP）的问题。为了解决CMDP问题与日志的反事实数据，我们提出了一个有效的学习算法，结合二分法搜索和基于模型的规划。首先，CMDP转换成其对偶使用拉格朗日松弛，这是证明是单调的对偶变量。此外，我们证明了对偶问题可以通过策略学习来解决，通过二分法搜索（即，通过利用单调性）。最后，我们表明，基于模型的规划可以用来有效地加快联合优化过程，而无需重新训练的政策，每一个双变量。合成和真实的营销数据集上的实证结果证实了我们的方法的有效性。
英文摘要：
Sequential incentive marketing is an important approach for online businesses to acquire customers, increase loyalty and boost sales. How to effectively allocate the incentives so as to maximize the return (e.g., business objectives) under the budget constraint, however, is less studied in the literature. This problem is technically challenging due to the facts that 1) the allocation strategy has to be learned using historically logged data, which is counterfactual in nature, and 2) both the optimality and feasibility (i.e., that cost cannot exceed budget) needs to be assessed before being deployed to online systems. In this paper, we formulate the problem as a constrained Markov decision process (CMDP). To solve the CMDP problem with logged counterfactual data, we propose an efficient learning algorithm which combines bisection search and model-based planning. First, the CMDP is converted into its dual using Lagrangian relaxation, which is proved to be monotonic with respect to the dual variable. Furthermore, we show that the dual problem can be solved by policy learning, with the optimal dual variable being found efficiently via bisection search (i.e., by taking advantage of the monotonicity). Lastly, we show that model-based planing can be used to effectively accelerate the joint optimization process without retraining the policy for every dual variable. Empirical results on synthetic and real marketing datasets confirm the effectiveness of our methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
