arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02583cs.SE
收藏

生成式软件工程

Generative Software Engineering

Yuan Huang, Yinan Chen, Xiangping Chen, Junqi Chen, Rui Peng, Zhicao Tang, Jinbo Huang, Furen Xu, Zibin Zheng

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文系统综述了预训练模型与大语言模型在软件工程生成式任务中的应用，分类梳理了相关模型、数据集和评估指标，并指出不足与未来研究方向。

摘要：

深度学习技术的快速发展、计算能力的提升以及大规模训练数据的可用性，推动了预训练模型和大语言模型（LLMs）的显著进步。基于BERT和Transformer等架构的预训练模型，以及ChatGPT等大语言模型，已经展现出卓越的语言能力，并在软件工程中得到了应用。软件工程任务可以划分为许多类别，其中生成式任务最受研究者关注。在这类任务中，预训练模型和大语言模型具备强大的语言表示能力和上下文感知能力，使其能够利用多样化的训练数据，并通过微调、迁移学习和提示工程来适应生成式任务。这些优势使它们成为生成式任务中的有效工具，并展现出优异的性能。本文对使用预训练模型和大语言模型进行软件工程生成式任务的研究进行了全面的文献综述。我们基于软件工程方法论对软件工程生成式任务进行了准确分类，并总结了所涉及的先进预训练模型和大语言模型，以及所使用的数据集和评估指标。此外，我们识别了现有方法的主要优势、不足和空白，并提出了潜在的研究方向。本综述旨在为研究人员和实践者提供关于预训练模型和大语言模型在软件工程生成式任务中应用的深入分析和指导。

英文摘要：

The rapid development of deep learning techniques, improved computational power, and the availability of vast training data have led to significant advancements in pre-trained models and large language models (LLMs). Pre-trained models based on architectures such as BERT and Transformer, as well as LLMs like ChatGPT, have demonstrated remarkable language capabilities and found applications in Software engineering. Software engineering tasks can be divided into many categories, among which generative tasks are the most concern by researchers, where pre-trained models and LLMs possess powerful language representation and contextual awareness capabilities, enabling them to leverage diverse training data and adapt to generative tasks through fine-tuning, transfer learning, and prompt engineering. These advantages make them effective tools in generative tasks and have demonstrated excellent performance. In this paper, we present a comprehensive literature review of generative tasks in SE using pre-trained models and LLMs. We accurately categorize SE generative tasks based on software engineering methodologies and summarize the advanced pre-trained models and LLMs involved, as well as the datasets and evaluation metrics used. Additionally, we identify key strengths, weaknesses, and gaps in existing approaches, and propose potential research directions. This review aims to provide researchers and practitioners with an in-depth analysis and guidance on the application of pre-trained models and LLMs in generative tasks within SE.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑