arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.11703cs.CL
收藏

利用大语言模型实现API交互:一个分类与合成数据生成框架

Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation

New York University(纽约大学);George Washington University(乔治华盛顿大学)

Chunliang Tao, Xiaojing Fan, Yahe Yang

AI总结:

本研究提出一个整合大语言模型的API交互框架,可实现自然语言到API调用的分类及定制化样本数据集自动生成,实验验证了其在模型测试选型中的有效性。

摘要:

随着大语言模型(LLMs)在自然语言处理领域不断发展,人们越来越关注借助其能力简化软件交互。本文提出了一个新颖的系统,该系统整合LLMs,既用于将自然语言输入分类为对应的API调用,也能自动创建适配特定API功能的样本数据集。通过对自然语言指令进行分类,我们的系统允许用户通过简单输入调用复杂的软件功能,提升了交互效率并降低了软件使用门槛。我们的数据集生成方法还能对不同LLM的API调用分类能力进行高效、系统的评估,为开发者或企业负责人评估LLM是否适用于定制化API管理提供了实用工具。我们针对多种API功能,使用生成的样本数据集在多个主流LLM上开展了实验。结果显示,GPT-4取得了0.996的高分类准确率,而LLaMA-3-8B的表现差得多,准确率仅为0.759。这些发现凸显了LLM变革API管理的潜力,也验证了我们的系统在指导不同应用场景下的模型测试与选型方面的有效性。

英文摘要:

As Large Language Models (LLMs) advance in natural language processing, there is growing interest in leveraging their capabilities to simplify software interactions. In this paper, we propose a novel system that integrates LLMs for both classifying natural language inputs into corresponding API calls and automating the creation of sample datasets tailored to specific API functions. By classifying natural language commands, our system allows users to invoke complex software functionalities through simple inputs, improving interaction efficiency and lowering the barrier to software utilization. Our dataset generation approach also enables the efficient and systematic evaluation of different LLMs in classifying API calls, offering a practical tool for developers or business owners to assess the suitability of LLMs for customized API management. We conduct experiments on several prominent LLMs using generated sample datasets for various API functions. The results show that GPT-4 achieves a high classification accuracy of 0.996, while LLaMA-3-8B performs much worse at 0.759. These findings highlight the potential of LLMs to transform API management and validate the effectiveness of our system in guiding model testing and selection across diverse applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑