arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.20144cs.CL
收藏

轨迹到任务:通过合成且可验证的数据训练鲁棒的工具调用代理以应对复杂用户意图

Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents

Northeastern University(东北大学);Amazon(亚马逊);Northwestern University(西北大学)

Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang

AI总结:

本文提出Trajectory2Task方法,通过合成可验证数据提升工具调用代理在复杂用户意图下的鲁棒性,通过多轮探索生成有效工具调用轨迹,并转换为可验证任务,最终在七种先进LLM上验证了改进效果。

摘要:

工具调用代理越来越多地应用于真实世界客户交互流程中。然而,大多数研究集中在理想化设置下的通用、固定和明确的任务上。在真实应用中,用户请求往往(1)模糊,(2)随时间变化,或(3)由于政策约束不可行,而覆盖这些多样复杂交互模式的训练和评估数据仍然不足。为弥合这一差距,我们提出了Trajectory2Task,一种可验证的数据生成管道,用于在三种现实用户场景下大规模研究工具使用:模糊意图、变化意图和不可行意图。该管道首先进行多轮探索以生成有效的工具调用轨迹。然后将这些轨迹转换为面向用户的任务,通过受控意图适应。此过程产生可验证的任务,支持闭环评估和训练。我们对七种最先进的LLM在生成的复杂用户场景任务上进行基准测试,并观察到频繁的失败。最后,使用从任务回放中获得的成功轨迹,我们对轻量级LLM进行微调,并在所有三种条件下发现一致的改进,同时在未见过的工具使用领域有更好的泛化能力,表明更强的工具调用能力。

英文摘要:

Tool-calling agents are increasingly deployed in real-world customer-facing workflows. Yet most studies on tool-calling agents focus on idealized settings with general, fixed, and well-specified tasks. In real-world applications, user requests are often (1) ambiguous, (2) changing over time, or (3) infeasible due to policy constraints, and training and evaluation data that cover these diverse, complex interaction patterns remain under-represented. To bridge the gap, we present Trajectory2Task, a verifiable data generation pipeline for studying tool use at scale under three realistic user scenarios: ambiguous intent, changing intent, and infeasible intents. The pipeline first conducts multi-turn exploration to produce valid tool-call trajectories. It then converts these trajectories into user-facing tasks with controlled intent adaptations. This process yields verifiable task that support closed-loop evaluation and training. We benchmark seven state-of-the-art LLMs on the generated complex user scenario tasks and observe frequent failures. Finally, using successful trajectories obtained from task rollouts, we fine-tune lightweight LLMs and find consistent improvements across all three conditions, along with better generalization to unseen tool-use domains, indicating stronger tool-calling ability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑