当标签忽略请求：审计合成对话式音乐推荐中策略选择的标签
When the Label Ignores the Request: Auditing Policy-Selected Targets in Synthetic Conversational Music Recommendation
浏览论文内容
中文总结 AI 辅助
本研究审计合成对话推荐中策略选择标签与用户确切歌曲请求的矛盾，发现半数开发轮次存在冲突，并通过训练时补充目录解析目标使nDCG@20相对提升53.3%。
中文摘要 AI 辅助
由大语言模型（LLM）流水线生成的合成对话现在被用作完整的对话式推荐基准：一个LLM听众与一个LLM推荐者交谈，对话中接下来记录的曲目成为每一轮对话的官方标签。这些策略选择的标签使得大规模评估具有可复现性，但它们只是模拟用户请求的代理。我们审计了标签与请求可直接比较的一个位置：用户按名称请求确切歌曲的轮次。在RecSys Challenge 2026 TalkPlay基准中，仅使用可见的对话和目录元数据，我们发现官方标签在审计的开发轮次中有一半与用户的确切歌曲请求相矛盾。这一问题不仅仅影响这一个基准：在真实音乐搜索中，指定所需项目是主导意图，部署的系统避免用替代品替换确切命名的项目，前提是这样做会损害满意度。一个小的训练时补充措施弥补了大部分差距：将目录解析的满足请求的目标添加到一小部分训练轮次中，在43个冲突轮次上nDCG@20相对提升了53.3%，同时保持官方指标不变，并通过匹配的对照组验证，该对照组检测相同的请求但仅使用官方标签进行训练。
英文摘要
Synthetic dialogues generated by LLM pipelines now serve as complete conversational-recommendation benchmarks: an LLM listener talks to an LLM recommender, and the track logged next in the conversation becomes the official label for each turn. These policy-selected labels make large-scale evaluation reproducible, but they are proxies for what the simulated user asked. We audit the one place where label and request are directly comparable: turns where the user asks for an exact song by name. In the RecSys Challenge 2026 TalkPlay benchmark, using visible dialogue and catalog metadata alone, we find that the official label contradicts the user's exact-song request in half of the audited development turns. This matters beyond one benchmark: naming the desired item is the dominant intent in real music search, where deployed systems avoid substituting an alternative for an exactly named item, on the premise that it costs satisfaction. A small training-time supplement closes most of the gap: adding catalog-resolved request-satisfying targets to a small fraction of training turns yields a 53.3% relative gain in nDCG@20 on the 43 conflict turns while leaving the official metric intact, verified against a matched control that detects the same requests but trains only on official labels.
