arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05040cs.CL
收藏

SWE-Fixer：训练开源大语言模型以实现有效且高效的 GitHub 问题解决

SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution

Shanghai AI Laboratory（上海人工智能实验室）；Xidian University（西安电子科技大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）

Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, Kai Chen

备注：
Our code, data, and model will be released at https://github.com/InternLM/SWE-Fixer
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出开源框架 SWE-Fixer，通过粗到细文件检索与代码编辑两模块高效解决 GitHub 问题，并在 SWE-Bench 基准上以极少模型调用取得领先性能。

摘要：

大语言模型（LLMs）已在多种复杂任务中展现出卓越的能力。LLM 的一个重要应用是应对软件工程挑战，特别是根据用户报告的问题修改代码，从而解决 GitHub 上的真实世界任务。然而，当前许多方法依赖闭源 LLM，这限制了可复现性、可访问性和透明度。LLM 解决软件工程问题的关键组成部分，以及如何有效增强其能力，仍不明确。为应对这些挑战，我们提出 SWE-Fixer，一个旨在有效且高效地解决 GitHub 问题的新型开源框架。SWE-Fixer 包含两个核心模块：代码文件检索模块和代码编辑模块。检索模块采用 BM25 与一个轻量级模型相结合，实现从粗到细的文件检索。随后，代码编辑模块利用另一个模型为已识别的文件生成补丁。为缓解公开可用数据集的不足，我们构建了一个包含 110K 个 GitHub 问题及其对应补丁的大规模数据集，并分别训练 SWE-Fixer 的两个模型。我们在 SWE-Bench Lite 和 Verified 基准上评估了我们的方法，在开源模型中取得了具有竞争力的性能，得分分别为 22.0% 和 30.2%。此外，通过 PASS_TO_PASS（P2P）过滤，SWE-Fixer 达到了最先进的性能（Lite 上 24.7%，Verified 上 32.8%）。另外，我们的方法每个实例仅需两次模型调用，使其显著比现有方法更高效。这些结果凸显了 SWE-Fixer 在真实世界代码修复场景中的有效性。我们将在 https://github.com/InternLM/SWE-Fixer 公开我们的模型、数据集和代码。

英文摘要：

Large Language Models (LLMs) have demonstrated remarkable proficiency across a variety of complex tasks. One significant application of LLMs is in tackling software engineering challenges, particularly in resolving real-world tasks on GitHub by fixing code based on the issues reported by the users. However, many current approaches rely on proprietary LLMs, which limits reproducibility, accessibility, and transparency. The critical components of LLMs for addressing software engineering issues and how their capabilities can be effectively enhanced remain unclear. To address these challenges, we introduce SWE-Fixer, a novel open-source framework designed to effectively and efficiently resolve GitHub issues. SWE-Fixer comprises two essential modules: a code file retrieval module and a code editing module. The retrieval module employs BM25 along with a lightweight model to achieve coarse-to-fine file retrieval. Subsequently, the code editing module utilizes the other model to generate patches for the identified files. To mitigate the lack of publicly available datasets, we compile an extensive dataset that includes 110K GitHub issues along with their corresponding patches and train the two models of SWE-Fixer separately. We assess our approach on the SWE-Bench Lite and Verified benchmarks, achieving competitive performance among open-source models with scores of 22.0% and 30.2%. Furthermore, SWE-Fixer reaches state-of-the-art performance (24.7% on Lite and 32.8% on Verified) with PASS_TO_PASS (P2P) filtering. Additionally, our approach requires only two model calls per instance, making it significantly more efficient than existing methods. These results highlight the effectiveness of SWE-Fixer in real-world code-fixing scenarios. We will make our model, dataset, and code publicly available at https://github.com/InternLM/SWE-Fixer.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑