arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.09293cs.CL
收藏

一个模型，多个目标：面向电商对话系统的自适应多目标学习

One Model, Multiple Goals: Adaptive Multi-Objective Learning for E-commerce Dialogue Systems

ByteDance（字节跳动）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Notre Dame（圣母大学）

Mingzhe Li, Jing Xiang, Enguo Zhou, Lang Gao, Tai Li, Qishen Zhang, Xiangliang Zhang, Xiuying Chen

备注：
Accepted by KDD 2026
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出自适应多目标强化学习框架MORE，通过将推理功能作为约束指导策略优化，并引入自适应多奖励机制平衡语言目标，在电商对话系统中同时提升推理准确性和语言自然性，在线实验转化率提升30.09%。

摘要：

电商场景中的对话系统通常需要满足多个目标：准确推理用户画像（如资格、信用额度）以确保正确决策和用户状态理解，同时生成自然且忠实的回复。这些目标是互补但非完全一致的。在这项工作中，我们提出了MORE，一个自适应多目标强化学习框架，联合优化推理准确性和语言自然性。我们的初步实验表明，直接混合具有不同优化动态的奖励会导致振荡和不稳定的学习。因此，我们不优化单一的混合奖励，而是将推理函数视为指导策略优化的约束。在推理时，系统直接生成回复，无需显式推理步骤，同时仍受益于推理增强的支架，避免额外的推理开销。为了更好地平衡回复生成过程中的语言目标，我们引入了一种自适应多奖励机制，该机制聚合流畅性和自然性等信号，并通过梯度反馈动态重新加权。我们在字节跳动的两个真实对话系统和MultiWOZ 2.2基准上评估MORE，其持续优于强基线。在字节跳动生产流量的14天在线实验中，MORE将总体转化率和达成转化率分别提高了16.53%和30.09%，同时提高了用户满意度并降低了转接率。值得注意的是，在人机对比中，MORE恢复了人类客服所实现的增量转化提升的约60%。

英文摘要：

Dialogue systems in e-commerce scenarios often need to satisfy multiple objectives: accurately reasoning over user profiles (e.g., eligibility, credit limit) to ensure correct decision-making and user state interpretation, while also generating natural and faithful responses. These goals are complementary but not identical. In this work, we propose MORE, an adaptive Multi-Objective REinforcement learning framework that jointly optimizes reasoning accuracy and linguistic naturalness. Our preliminary experiments show that directly mixing rewards with diverging optimization dynamics can cause oscillations and unstable learning. Thus, instead of optimizing a single mixed reward, we treat reasoning functions as constraints that guide policy optimization. At inference time, the system directly generates responses without explicit reasoning steps, while still benefiting from reasoning-enhanced scaffold and avoiding additional inference overhead. To better balance linguistic objectives during response generation, we introduce an adaptive multi-reward mechanism that aggregates signals such as fluency and naturalness and dynamically reweighs them via gradient feedback. We evaluate MORE on two real-world dialogue systems at ByteDance and the MultiWOZ 2.2 benchmark, where it consistently outperforms strong baselines. In 14-day online experiments on ByteDance production traffic, MORE improves overall and reached conversion by 16.53% and 30.09%, while increasing user satisfaction and reducing handoff rates. Notably, in a human-machine comparison, MORE recovers about 60% of the incremental conversion lift achieved by human agents.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑