arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.02458cs.LGcs.AI
收藏

基于多目标决策Transformer的生成式序列通知优化

Generative Sequential Notification Optimization via Multi-Objective Decision Transformers

LinkedIn Mountain View USA(LinkedIn Mountain View)

Borja Ocejo, Ruofan Wang, Ke Liu, Rohit K. Patra, Haotian Shen, David Liu, Yiwen Yuan, Gokulraj Mohanasundaram, Fedor Borisyuk, Prakruthi Prabhakar

AI总结:

针对通知优化序列决策难题,提出基于决策Transformer的多目标优化框架,重构策略学习为回报条件监督学习,经实验验证可提升通知效用与会话活跃度、降低用户疲劳,在LinkedIn场景中效果优于CQL方法。

摘要:

通知是传递及时且相关信息的重要通信渠道。优化通知推送需要在消息效用、用户疲劳度等约束下,解决复杂的序列决策难题。保守Q学习(Conservative Q-Learning, CQL)等离线强化学习(RL)方法已被应用于该问题,但在规模化应用中面临诸多实际挑战,包括不稳定性、对分布偏移的敏感性、可复现性有限,以及高维推荐场景下的可解释性难题。我们提出了一个基于决策Transformer(Decision Transformer, DT)的框架,将策略学习重构为基于回报条件的监督学习,提升了鲁棒性、可扩展性和建模灵活性。我们的贡献包括:与CQL的真实场景对比、适用于非回合式任务的多奖励设计、用于未来回报条件化的分位数回归方法,以及配备基于循环缓冲区序列处理、支持近实时推理的可投产系统。在已部署的通知系统中开展的大量离线和在线实验表明,我们的方法在降低用户疲劳度的同时,提升了通知效用和整体会话活跃度。与基于多目标CQL的智能体相比,该基于DT的方法通过提升通知推荐的相关性,使LinkedIn的通知决策相关会话数提升了0.72%。

英文摘要:

Notifications are an important communication channel for delivering timely and relevant information. Optimizing their delivery involves addressing complex sequential decision-making challenges under constraints such as message utility and user fatigue. Offline reinforcement learning (RL) methods, such as Conservative Q-Learning (CQL), have been applied to this problem but face practical challenges at scale, including instability, sensitivity to distribution shifts, limited reproducibility, and difficulties with explainability in high-dimensional recommendation settings. We present a Decision Transformer (DT) based framework that reframes policy learning as return-conditioned supervised learning, improving robustness, scalability, and modeling flexibility. Our contributions include a real-world comparison with CQL, a multi-reward design suitable for non-episodic tasks, a quantile regression approach to return-to-go conditioning, and a production-ready system with circular buffer-based sequence processing for near-real-time inference. Extensive offline and online experiments in a deployed notification system show that our approach improves notification utility and overall session activity while minimizing user fatigue. Compared to a multi-objective CQL-based agent, the DT-based approach achieved a +0.72% increase in sessions for notification decision-making at LinkedIn by making notification recommendation more relevant.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑