RLTP：面向预加载广告中延迟曝光建模的强化学习节奏控制
RLTP: Reinforcement Learning to Pace for Delayed Impression Modeling in Preloaded Ads
AI总结：
针对预加载广告的延迟曝光导致传统节奏控制失效的问题，提出强化学习框架 RLTP，通过顺序决策与定制奖励联合保证曝光量并优化投放效果，实验和线上部署均显示显著提升。
摘要：
为提升品牌认知度，许多广告主与广告平台签订合同以购买流量，然后向目标受众投放广告。在整个投放周期内，广告主通常希望广告获得一定的曝光次数，同时也期望投放表现尽可能好（例如获得较高的点击率）。广告平台采用节奏控制算法，通过实时调整对流量请求的选择概率来满足这些需求。然而，投放过程还受到发布方策略的影响，而这些策略是广告平台无法控制的。预加载是许多广告类型（例如视频广告）广泛使用的策略，用于确保流量请求后展示广告的响应时间合理，但这会导致延迟曝光现象。传统节奏控制算法无法很好地处理预加载特性，因为它们依赖即时反馈信号，可能无法保证满足广告主的需求。本文聚焦于预加载广告曝光节奏控制这一新研究问题，并提出了一个强化学习节奏控制框架 RLTP（Reinforcement Learning To Pace）。该框架学习一个节奏控制智能体，在整个投放周期内按顺序生成选择概率。为了联合优化曝光次数和投放表现这两个目标，RLTP 采用定制化的奖励估计器，以保证曝光次数、惩罚超投并最大化流量价值。在大规模工业数据集上的实验验证了 RLTP 相比基线节奏控制算法具有显著优势。我们已将 RLTP 框架在线部署到广告平台，结果表明其对投放完成率和点击率等核心指标实现了显著提升。
英文摘要：
To increase brand awareness, many advertisers conclude contracts with advertising platforms to purchase traffic and then deliver advertisements to target audiences. In a whole delivery period, advertisers usually desire a certain impression count for the ads, and they also expect that the delivery performance is as good as possible (e.g., obtaining high click-through rate). Advertising platforms employ pacing algorithms to satisfy the demands via adjusting the selection probabilities to traffic requests in real-time. However, the delivery procedure is also affected by the strategies from publishers, which cannot be controlled by advertising platforms. Preloading is a widely used strategy for many types of ads (e.g., video ads) to make sure that the response time for displaying after a traffic request is legitimate, which results in delayed impression phenomenon. Traditional pacing algorithms cannot handle the preloading nature well because they rely on immediate feedback signals, and may fail to guarantee the demands from advertisers. In this paper, we focus on a new research problem of impression pacing for preloaded ads, and propose a Reinforcement Learning To Pace framework RLTP. It learns a pacing agent that sequentially produces selection probabilities in the whole delivery period. To jointly optimize the two objectives of impression count and delivery performance, RLTP employs tailored reward estimator to satisfy the guaranteed impression count, penalize the over-delivery and maximize the traffic value. Experiments on large-scale industrial datasets verify that RLTP outperforms baseline pacing algorithms by a large margin. We have deployed the RLTP framework online to our advertising platform, and results show that it achieves significant uplift to core metrics including delivery completion rate and click-through rate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
