arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.02479cs.SEcs.AIcs.CL
收藏

从大型语言模型（LLMs）到基于LLM的智能体：软件工程领域的现状、挑战与未来综述

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

School of Electrical and Computer Engineering, The University of Sydney（悉尼大学电气与计算机工程学院）；University at Buffalo, SUNY（纽约州立大学布法罗分校）；University of Chicago（芝加哥大学）

Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文综述LLMs与基于LLM的智能体在软件工程中的应用现状，区分二者差异，分析六个关键主题的相关工作及模型基准，为该领域未来研究提供启示。

摘要：

随着大型语言模型（LLMs）的兴起，研究人员正越来越多地探索其在软件工程等各个垂直领域的应用。LLMs在代码生成和漏洞检测等领域取得了显著成功，但也存在诸多局限和不足。基于LLM的智能体是一种具有通用人工智能（AGI）潜力的新技术，它将LLMs作为决策和行动的核心，解决了LLMs固有的一些局限性，如缺乏自主性和自我改进能力。尽管已有大量研究和综述探讨LLMs在软件工程中的应用可能性，但缺乏对LLMs和基于LLM的智能体的明确区分。目前，在软件工程领域中，将LLM解决方案认定为基于LLM的智能体的统一标准和基准测试仍处于早期阶段。在本综述中，我们广泛调查了LLMs和基于LLM的智能体在软件工程中的当前实践和解决方案，特别总结了六个关键主题：需求工程、代码生成、自主决策、软件设计、测试生成和软件维护。我们从这六个主题出发，回顾并区分了LLMs和基于LLM的智能体的相关工作，分析了它们在任务、基准和评估指标方面的异同。最后，我们讨论了所使用的模型和基准，对其在软件工程中的应用和有效性进行了全面分析。我们期望这项工作能为推动基于LLM的智能体在软件工程领域的未来研究提供启示。

英文摘要：

With the rise of large language models (LLMs), researchers are increasingly exploring their applications in var ious vertical domains, such as software engineering. LLMs have achieved remarkable success in areas including code generation and vulnerability detection. However, they also exhibit numerous limitations and shortcomings. LLM-based agents, a novel tech nology with the potential for Artificial General Intelligence (AGI), combine LLMs as the core for decision-making and action-taking, addressing some of the inherent limitations of LLMs such as lack of autonomy and self-improvement. Despite numerous studies and surveys exploring the possibility of using LLMs in software engineering, it lacks a clear distinction between LLMs and LLM based agents. It is still in its early stage for a unified standard and benchmarking to qualify an LLM solution as an LLM-based agent in its domain. In this survey, we broadly investigate the current practice and solutions for LLMs and LLM-based agents for software engineering. In particular we summarise six key topics: requirement engineering, code generation, autonomous decision-making, software design, test generation, and software maintenance. We review and differentiate the work of LLMs and LLM-based agents from these six topics, examining their differences and similarities in tasks, benchmarks, and evaluation metrics. Finally, we discuss the models and benchmarks used, providing a comprehensive analysis of their applications and effectiveness in software engineering. We anticipate this work will shed some lights on pushing the boundaries of LLM-based agents in software engineering for future research.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑