arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.16443cs.CLcs.AIcs.SE
收藏

CodeS:通过多层草图实现自然语言到代码仓库

CodeS: Natural Language to Code Repository via Multi-Layer Sketch

Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学);Huawei Technologies Co., Ltd(华为技术有限公司);Chinese Academy of Sciences(中国科学院);Shandong University(山东大学);Funcun-wuyou Technologies Co., Ltd(Funcun-wuyou科技有限公司)

Daoguang Zan, Ailun Yu, Wei Liu, Dong Chen, Bo Shen, Wei Li, Yafen Yao, Yongshun Gong, Xiaolin Chen, Bei Guan, Zhiguang Yang, Yongji Wang, Qianxiang Wang, Lizhen Cui

AI总结:

提出自然语言到代码仓库(NL2Repo)任务及框架CodeS,通过多层草图将其分解为生成目录结构、文件草图和函数细节三个子任务,并构建基准与插件验证了其有效性与实用性。

摘要:

大型语言模型(LLMs)在代码相关任务上的出色表现展示了完全自动化软件开发的潜力。有鉴于此,我们引入了一项新的软件工程任务,即自然语言到代码仓库(NL2Repo)。该任务旨在根据自然语言需求生成整个代码仓库。为了解决这一任务,我们提出了一个简单而有效的框架CodeS,它通过多层草图将NL2Repo分解为多个子任务。具体而言,CodeS包含三个模块:RepoSketcher、FileSketcher和SketchFiller。RepoSketcher首先根据给定需求生成仓库的目录结构;FileSketcher随后为生成结构中的每个文件生成文件草图;SketchFiller最后为生成的文件草图中的每个函数填充细节。为了在NL2Repo任务上严格评估CodeS,我们通过自动化基准测试和人工反馈分析进行了评估。对于基于基准的评估,我们构建了一个面向仓库的基准SketchEval,并设计了一个评估指标SketchBLEU。对于基于反馈的评估,我们为CodeS开发了VSCode插件,并招募了30名参与者开展实证研究。大量实验证明了CodeS在NL2Repo任务上的有效性和实用性。

英文摘要:

The impressive performance of large language models (LLMs) on code-related tasks has shown the potential of fully automated software development. In light of this, we introduce a new software engineering task, namely Natural Language to code Repository (NL2Repo). This task aims to generate an entire code repository from its natural language requirements. To address this task, we propose a simple yet effective framework CodeS, which decomposes NL2Repo into multiple sub-tasks by a multi-layer sketch. Specifically, CodeS includes three modules: RepoSketcher, FileSketcher, and SketchFiller. RepoSketcher first generates a repository's directory structure for given requirements; FileSketcher then generates a file sketch for each file in the generated structure; SketchFiller finally fills in the details for each function in the generated file sketch. To rigorously assess CodeS on the NL2Repo task, we carry out evaluations through both automated benchmarking and manual feedback analysis. For benchmark-based evaluation, we craft a repository-oriented benchmark, SketchEval, and design an evaluation metric, SketchBLEU. For feedback-based evaluation, we develop a VSCode plugin for CodeS and engage 30 participants in conducting empirical studies. Extensive experiments prove the effectiveness and practicality of CodeS on the NL2Repo task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑