arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.04464cs.SEcs.AIcs.LG
收藏

论推理在仓库级代码编辑中上下文检索的重要性

On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing

JetBrains Research(JetBrains研究院)

Alexander Kovrigin, Aleksandra Eliseeva, Yaroslav Zharov, Timofey Bryksin

AI总结:

本文将仓库级代码编辑中的上下文检索与流水线其他组件解耦,通过专门实验分析推理的作用,发现推理能提升上下文精确度但无法判断上下文充分性,并概述了专用工具在上下文收集中的最终角色。

摘要:

近年来,具备代码流畅性的大语言模型(LLMs)的进展使仓库级代码编辑研究成为可能。在这类任务中,模型需要根据请求导航并修改项目的整个代码库。因此,此类任务需要高效的上下文检索,即在庞大的代码库中导航以收集相关上下文。尽管上下文检索的重要性已得到公认,现有研究往往以端到端的方式处理仓库级编码任务,导致这些复杂系统中各个组件的影响难以明确。在本工作中,我们将上下文检索任务与仓库级代码编辑流水线中的其他组件解耦。通过专注于上下文检索的实验,我们为界定该组件的优势与不足以及推理在其中所起的作用奠定了基础。我们得出结论:虽然推理有助于提高所收集上下文的精确度,但它仍然缺乏识别上下文充分性的能力。我们还概述了专用工具在上下文收集过程中的最终角色。本文的补充代码可在 https://github.com/JetBrains-Research/ai-agents-code-editing 获取。

英文摘要:

Recent advancements in code-fluent Large Language Models (LLMs) enabled the research on repository-level code editing. In such tasks, the model navigates and modifies the entire codebase of a project according to request. Hence, such tasks require efficient context retrieval, i.e., navigating vast codebases to gather relevant context. Despite the recognized importance of context retrieval, existing studies tend to approach repository-level coding tasks in an end-to-end manner, rendering the impact of individual components within these complicated systems unclear. In this work, we decouple the task of context retrieval from the other components of the repository-level code editing pipelines. We lay the groundwork to define the strengths and weaknesses of this component and the role that reasoning plays in it by conducting experiments that focus solely on context retrieval. We conclude that while the reasoning helps to improve the precision of the gathered context, it still lacks the ability to identify its sufficiency. We also outline the ultimate role of the specialized tools in the process of context gathering. The code supplementing this paper is available at https://github.com/JetBrains-Research/ai-agents-code-editing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑