arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.28119cs.SE
收藏

基于LLM的问题解决的代码上下文压缩

Compressing Code Context for LLM-based Issue Resolution

Peking University(北京大学);University College London(伦敦大学学院)

Haoxiang Jia, Earl T. Barr, Sergey Mechtaev

AI总结:

本文提出OCD算法和SWEzze模型,通过遗传搜索和delta调试压缩代码上下文,提升问题解决效率,实现6倍压缩率,减少51.8%-71.3%的token预算,提高5.0%-9.2%的问题解决率。

摘要:

大型语言模型(LLM)现在能够解决现实中的GitHub问题。然而,当前方法过度逼近代码上下文,并面临两个叠加问题:处理大量输入的高昂成本,以及噪声涌入上下文窗口导致模型难以聚焦修复信号。现有压缩技术无法解决这一矛盾:通用压缩器牺牲了代码的语义完整性,而代码专用工具缺乏对代码结构和任务上下文的认知,无法保留关键补丁成分。为此,我们提出一个包含两个组件的新型框架。首先,Oracle-guided Code Distillation(OCD),一种结合遗传搜索和delta调试的上下文蒸馏算法,系统地将代码上下文减少到其最小必要子序列,仅保留修复成功的必要成分。我们使用蒸馏后的数据对SWEzze进行微调,这是一种轻量级模型,学会在推理时压缩代码上下文,过滤噪声并对抗干扰,同时保留修复成分。在SWE-bench Verified上评估了三个前沿LLM,SWEzze在模型间保持约6倍的压缩率,相对于未压缩设置减少了51.8%-71.3%的总token预算,提高了5.0%-9.2%的问题解决率,并在有效性、压缩率和延迟之间提供了最佳平衡,优于最先进的上下文压缩基线。

英文摘要:

Large Language Models (LLMs) are now capable of resolving real-world GitHub issues. However, current approaches overapproximate the code context and suffer from two compounding problems: the prohibitive cost of processing massive inputs, and low effectiveness as noise floods the context window and distracts the model from the bug-fixing signal. Existing compression techniques fail to resolve this tension: generic compressors compromise the semantic integrity of code, while code-specific tools lack awareness of code structure and task context to preserve essential patch ingredients. To address this, we propose a novel framework consisting of two components. First, Oracle-guided Code Distillation (OCD), a context distillation algorithm that combines genetic search and delta debugging to systematically reduce code contexts to their minimal sufficient subsequence - retaining only the ingredients required for a successful fix. We use this distilled data to fine-tune SWEzze, a lightweight model that learns to compress code context at inference time, filtering noise and combating distraction while preserving fix ingredients. Evaluated on SWE-bench Verified across three frontier LLMs, SWEzze maintains a stable compression rate of about 6 times across models, reduces the total token budget by 51.8%-71.3% relative to the uncompressed setting, improves issue resolution rates by 5.0%-9.2%, and delivers the best overall balance among effectiveness, compression ratio, and latency compared with state-of-the-art context compression baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑