arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22697cs.CLcs.AI
收藏

强化现实服务代理：在任务导向对话中平衡效用与成本

Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue

Ning Gao；Wei Zhang；Yuqin Dai；Ling Shi；Ziyin Wang；Yujie Wang；Wei He；Jinpeng Wang；Chaozheng Wang

Ning Gao, Wei Zhang, Yuqin Dai, Ling Shi, Ziyin Wang, Yujie Wang, Wei He, Jinpeng Wang, Chaozheng Wang

备注：
35 pages, 8 tables, 3 figures
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

InteractCS-RL通过多粒度强化学习框架，在任务导向对话中平衡效用与成本，通过用户驱动策略和成本感知优化提升代理性能。

摘要：

大型语言模型（LLMs）的快速演进加速了从对话聊天机器人向通用代理的转变。然而，有效平衡同理心沟通与预算感知决策仍是一个开放性挑战。由于现有方法无法捕捉这些复杂的策略权衡，我们提出了InteractCS-RL框架，将任务导向对话重新定义为多粒度强化学习过程。具体而言，我们首先建立了以用户为中心的交互框架，提供高保真的训练环境，使代理能够动态探索多样化的策略，以人格驱动的用户为指导。然后，我们引入了具有混合优势估计策略的成本感知多轮策略优化（CMPO）。通过整合生成过程信用并采用PID-拉格朗日成本控制器，CMPO有效引导策略在用户奖励和全局成本约束之间探索帕累托边界。在定制的现实商业场景上的大量实验表明，InteractCS-RL在三个评估维度上显著优于其他基线。进一步在工具-代理-用户交互基准上的评估验证了InteractCS-RL在不同领域中的鲁棒性。

英文摘要：

The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑