arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01940cs.AI
收藏

CoVe:通过约束引导验证训练交互工具使用代理

CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification

Huawei Research(华为研究)

Jinpeng Chen, Cheng Gong, Hanbo Li, Ziru Liu, Zichen Tian, Xinyu Fu, Shi Wu, Chenyang Zhang, Wu Zhang, Suiyun Zhang, Dandan Tu, Rui Liu

AI总结:

CoVe通过约束引导验证方法有效提升交互式工具使用代理的训练数据合成效率与质量。

摘要:

开发多轮交互式工具使用代理具有挑战性,因为现实世界用户需求往往复杂且模糊,但代理必须执行确定性动作来满足这些需求。为解决这一差距,我们引入了CoVe(约束-验证),一种后训练数据合成框架,旨在在确保数据复杂性和正确性的前提下训练交互式工具使用代理。CoVe首先定义显式的任务约束,这些约束具有双重作用:它们指导复杂轨迹的生成,并作为确定性验证器来评估轨迹质量。这使得能够为监督微调(SFT)生成高质量的训练轨迹,并推导出准确的强化学习(RL)奖励信号。我们在具有挑战性的τ²-bench基准上评估了该框架的有效性。值得注意的是,我们的紧凑型CoVe-4B模型在航空和零售领域分别达到了43.0%和59.4%的成功率;其整体性能显著优于相似规模的强大基线,并且与规模达17倍的模型具有竞争力。这些结果表明,CoVe为合成最先进的交互式工具使用代理的训练数据提供了一条有效且高效的途径。为了支持未来研究,我们开源了我们的代码、训练模型以及用于训练的完整12000个高质量轨迹集。

英文摘要:

Developing multi-turn interactive tool-use agents is challenging because real-world user needs are often complex and ambiguous, yet agents must execute deterministic actions to satisfy them. To address this gap, we introduce \textbf{CoVe} (\textbf{Co}nstraint-\textbf{Ve}rification), a post-training data synthesis framework designed for training interactive tool-use agents while ensuring both data complexity and correctness. CoVe begins by defining explicit task constraints, which serve a dual role: they guide the generation of complex trajectories and act as deterministic verifiers for assessing trajectory quality. This enables the creation of high-quality training trajectories for supervised fine-tuning (SFT) and the derivation of accurate reward signals for reinforcement learning (RL). Our evaluation on the challenging $τ^2$-bench benchmark demonstrates the effectiveness of the framework. Notably, our compact \textbf{CoVe-4B} model achieves success rates of 43.0\% and 59.4\% in the Airline and Retail domains, respectively; its overall performance significantly outperforms strong baselines of similar scale and remains competitive with models up to $17\times$ its size. These results indicate that CoVe provides an effective and efficient pathway for synthesizing training data for state-of-the-art interactive tool-use agents. To support future research, we open-source our code, trained model, and the full set of 12K high-quality trajectories used for training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑