在Spotify引导会话推荐代理:合成数据生成与自我改进循环
Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops
- Spotify
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
本文提出合成数据生成与自我改进循环方法,优化会话推荐代理的规划与工具调用,在Spotify生产中提升质量8%,A/B测试显示收听量增14%、周活跃用户增5%、跳过率降5%。
AI中文摘要:
会话推荐代理是内容发现的一种新范式,使用户能够通过自然语言表达复杂意图(例如,“推荐我未曾听过的意大利独立艺术家”)。构建此类代理的一个核心挑战是优化代理规划——即决定如何选择、排序和调用工具——尤其是在冷启动场景下,此时尚无真实用户交互可用。我们引入了一个多轮合成数据生成流水线和自我改进循环来解决这一挑战。合成数据流水线将单轮提示转换为逼真的多轮对话,从而在发布前实现系统性评估。自我改进循环结合了基于方差的对比优化与通过编码代理进行的迭代细化,自动识别并修复规划和工具使用错误。我们的方法在高度优化的手动提示基础上将质量提升了+8%。该系统已投入生产,并显著加速了Spotify会话推荐代理发布的迭代周期。在线A/B测试证明了其有效性,与仅支持会话细化的先前体验相比,用户收听量增加了+14%,周活跃用户增加了+5%,跳过率降低了5%。这项工作为加速行业中会话推荐代理的开发提供了一个实用框架。
英文摘要:
Conversational recommendation agents are a new paradigm for content discovery, enabling users to express complex intents through natural language (e.g., "recommend Italian indie artists I haven't heard before"). A central challenge in building such agents is optimizing agent planning -- deciding how to select, sequence, and invoke tools -- particularly in cold-start settings where real user interactions are not yet available. We introduce a pipeline for multi-turn synthetic data generation and a self-improvement loop to address this challenge. The synthetic data pipeline transforms single-turn prompts into realistic multi-turn conversations, enabling systematic evaluation before launch. The self-improvement loop combines variance-based contrastive optimization with iterative refinement through a coding agent, automatically identifying and fixing planning and tool-use errors. Our approach improves quality by +8% on top of a highly optimized manual prompt. The system has been productionized and significantly accelerated iteration cycles for the launch of a conversational recommendation agent at Spotify. Online A/B tests demonstrate its effectiveness, with +14% user listening, +5% increase in weekly active users, and a 5% reduction in skip rate compared to a prior experience supporting only session refinement. This work provides a practical framework for accelerating the development of conversational recommendation agents in industry.