arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.19487cs.SE
收藏

RLCoder:用于仓库级代码补全的强化学习

RLCoder: Reinforcement Learning for Repository-Level Code Completion

Yanlin Wang, Yanli Wang, Daya Guo, Jiachi Chen, Ruikai Zhang, Yuchi Ma, Zibin Zheng

AI总结:

针对仓库级代码补全中检索方法语义捕捉不足及缺乏标注数据的问题,提出强化学习框架 RLCoder,通过基于困惑度的迭代反馈机制训练检索器并引入停止信号机制,在多个基准上取得显著性能提升。

摘要:

仓库级代码补全旨在为指定仓库上下文中的未完成代码片段生成代码。由于输入序列长度的限制,现有方法主要依赖检索增强生成策略。然而,像 BM25 这样的传统基于词法的检索方法难以捕捉代码语义,而基于模型的检索方法由于缺乏用于训练的标注数据而面临挑战。因此,我们提出了 RLCoder,这是一个新颖的强化学习框架,它能够让检索器学会在不需要标注数据的情况下检索对代码补全有用的内容。具体而言,我们基于将检索到的内容作为额外上下文时目标代码的困惑度,迭代地评估检索内容的有用性,并提供反馈以更新检索器参数。这种迭代过程使检索器能够从其成功和失败中学习,逐渐提高其检索相关且高质量内容的能力。考虑到并非所有情况都需要代码文件之外的信息,且并非所有检索到的上下文都有助于生成,我们还引入了一种停止信号机制,允许检索器自主决定何时检索以及保留哪些候选内容。大量实验结果表明,RLCoder 在 CrossCodeEval 和 RepoEval 上一致优于最先进的方法,实现了比以前方法 12.2% 的 EM 提升。此外,实验表明我们的框架能够泛化到不同的编程语言,并进一步改进了以前的方法,如 RepoCoder。我们在 https://github.com/DeepSoftwareAnalytics/RLCoder 提供了代码和数据。

英文摘要:

Repository-level code completion aims to generate code for unfinished code snippets within the context of a specified repository. Existing approaches mainly rely on retrieval-augmented generation strategies due to limitations in input sequence length. However, traditional lexical-based retrieval methods like BM25 struggle to capture code semantics, while model-based retrieval methods face challenges due to the lack of labeled data for training. Therefore, we propose RLCoder, a novel reinforcement learning framework, which can enable the retriever to learn to retrieve useful content for code completion without the need for labeled data. Specifically, we iteratively evaluate the usefulness of retrieved content based on the perplexity of the target code when provided with the retrieved content as additional context, and provide feedback to update the retriever parameters. This iterative process enables the retriever to learn from its successes and failures, gradually improving its ability to retrieve relevant and high-quality content. Considering that not all situations require information beyond code files and not all retrieved context is helpful for generation, we also introduce a stop signal mechanism, allowing the retriever to decide when to retrieve and which candidates to retain autonomously. Extensive experimental results demonstrate that RLCoder consistently outperforms state-of-the-art methods on CrossCodeEval and RepoEval, achieving 12.2% EM improvement over previous methods. Moreover, experiments show that our framework can generalize across different programming languages and further improve previous methods like RepoCoder. We provide the code and data at https://github.com/DeepSoftwareAnalytics/RLCoder.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑