arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.18518cs.CLcs.AIcs.LGcs.SE
收藏

APIGen:用于生成可验证且多样化的函数调用数据集的自动化流水线

APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets

Salesforce AI Research(Salesforce AI研究院)
Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio … 展开作者

Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, Caiming Xiong

AI总结:

APIGen提出自动化数据生成流水线,利用3,673个可执行API生成可验证的多样化函数调用数据集,经三级验证确保质量,7B参数模型在伯克利函数调用基准上超越GPT-4,并发布60,000条高质量数据。

摘要:

函数调用智能体模型的进步需要多样化、可靠且高质量的数据集。本文提出了APIGen,一种自动化数据生成流水线,旨在为函数调用应用合成可验证的高质量数据集。我们利用APIGen收集了21个不同类别的3,673个可执行API,以可扩展且结构化的方式生成多样化的函数调用数据集。我们数据集中的每条数据都通过三个层级阶段进行验证:格式检查、实际函数执行和语义验证,确保其可靠性和正确性。我们证明,使用我们精选数据集训练的模型,即使仅有7B参数,也能在伯克利函数调用基准上实现最先进的性能,超越多个GPT-4模型。此外,我们的1B模型表现出卓越性能,超越了GPT-3.5-Turbo和Claude-3 Haiku。我们发布了一个包含60,000条高质量条目的数据集,旨在推动函数调用智能体领域的发展。该数据集可在Huggingface上获取:https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k,项目主页:https://apigen-pipeline.github.io/

英文摘要:

The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑