arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.17829cs.CLcs.AI
收藏

合成数据中语言和论证多样性用于功能调用代理

Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agents

Technion(技术学院);TII(技术研究所)

Dan Greenstein, Zohar Karnin, Chen Amiraz, Oren Somekh

AI总结:

本文提出了一种生成合成数据集的方法,通过优化通用多样性度量标准,提高功能调用代理训练数据的语言和论证多样性,从而提升模型在分布外任务中的性能。

摘要:

功能调用代理的构建已成为扩展模型能力的一个有前途的领域。这项任务的主要挑战是获得高质量且多样化的数据进行训练。先前的工作强调了函数、调用模式和交互轮次的多样性,但请求的语言多样性以及论证的覆盖范围(例如,city_name、stock_ticker)仍处于未探索的领域。我们提出了一种方法,通过优化通用多样性度量标准来生成合成数据集,既包括查询又包括论证,而无需依赖人工制定的规则或分类学,使其在不同使用场景下都具有鲁棒性。我们通过内在和外在测试验证了我们技术的有效性,将其与最先进的数据生成方法进行比较。我们展示了在多样性方面优于基线,同时保持可比的正确性。此外,当用作训练集时,由我们的数据集生成的模型在分布外性能方面优于基于基线数据生成方法的类似模型。特别是,我们在BFCL基准测试中相比类似对手实现了7.4%的准确率提升。

英文摘要:

The construction of function calling agents has emerged as a promising avenue for extending model capabilities. A major challenge for this task is obtaining high quality diverse data for training. Prior work emphasizes diversity in functions, invocation patterns, and interaction turns, yet linguistic diversity of requests and coverage of arguments (e.g., \texttt{city\_name}, \texttt{stock\_ticker}) remain underexplored. We propose a method that generates synthetic datasets via optimizing general-purpose diversity metrics across both queries and arguments, without relying on hand-crafted rules or taxonomies, making it robust to different usecases. We demonstrate the effectiveness of our technique via both intrinsic and extrinsic testing, comparing it to SoTA data generation methods. We show a superiority over baselines in terms of diversity, while keeping comparable correctness. Additionally, when used as a training set, the model resulting from our dataset exhibits superior performance compared to analogous models based on the baseline data generation methods in out-of-distribution performance. In particular, we achieve an $7.4\%$ increase in accuracy on the BFCL benchmark compared to similar counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑