arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.02534cs.CL
收藏

面向机器学习库开发的自适应自改进LLM智能体系统

Adaptive Self-improvement LLM Agentic System for ML Library Development

Department of Computer Science, Stanford University, USA(计算机科学系,斯坦福大学)

Genghan Zhang, Weixin Liang, Olivia Hsu, Kunle Olukotun

AI总结:

本文提出一种自适应自改进LLM智能体系统,通过在数据有限的架构专用编程语言场景下进行复杂推理,使机器学习库代码生成性能较单一LLM基线最高提升3.9倍。

摘要:

机器学习库通常使用面向特定领域架构的架构专用编程语言(ASPL)编写,是高效机器学习系统的关键。然而,编写这些高性能机器学习库具有挑战性,因为它需要机器学习算法和ASPL方面的专业知识。另一方面,大语言模型(LLM)已展现出通用的编码能力。然而,使用LLM通过ASPL生成机器学习库仍然面临挑战,因为1)即使对于经验丰富的人类程序员来说,这项任务也很复杂;2)由于ASPL深奥且不断演进的特性,可用的代码示例十分有限。因此,LLM需要在数据有限的情况下进行复杂推理才能完成这项任务。为应对这些挑战,我们引入了一种自适应自改进智能体系统。为了评估我们系统的有效性,我们构建了一个典型机器学习库的基准,并在此基准上使用开源和闭源LLM生成ASPL代码。我们的结果表明,与单一LLM基线相比,性能提升最高可达$3.9\ imes$。

英文摘要:

ML libraries, often written in architecture-specific programming languages (ASPLs) that target domain-specific architectures, are key to efficient ML systems. However, writing these high-performance ML libraries is challenging because it requires expert knowledge of ML algorithms and the ASPL. Large language models (LLMs), on the other hand, have shown general coding capabilities. However, challenges remain when using LLMs for generating ML libraries using ASPLs because 1) this task is complicated even for experienced human programmers and 2) there are limited code examples because of the esoteric and evolving nature of ASPLs. Therefore, LLMs need complex reasoning with limited data in order to complete this task. To address these challenges, we introduce an adaptive self-improvement agentic system. In order to evaluate the effectiveness of our system, we construct a benchmark of a typical ML library and generate ASPL code with both open and closed-source LLMs on this benchmark. Our results show improvements of up to $3.9\times$ over a baseline single LLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑