发表机构
ITMO University(ITMO大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
本研究在IssueLoc-Bench上评估5种探索器模型,发现低成本探索器可保留较高定位质量并大幅降低耗时与token用量,支持将仓库探索作为编码智能体的模块化独立阶段。
AI 中文摘要
代码仓库探索是编码智能体流程中一个独特且成本高昂的阶段:在生成补丁前,智能体必须识别出哪些代码仓库文件可能相关。本研究探讨是否可将该阶段委托给成本更低的模型,同时保持可用的定位质量。我们使用IssueLoc-Bench(问题定位基准),在相同的只读交互界面下评估5种探索器模型,涉及499个来自SWE-bench Verified(软件工程基准验证集)的任务,以及来自153个额外代码仓库的500个任务。我们衡量早期候选文件发现、前三黄金文件覆盖率、严格文件集恢复率、智能体耗时和token使用量,并开展成对实例级不确定性分析和按代码仓库聚类的敏感性分析。质量最高的探索器在所有定位指标中表现最优,但也存在显著更经济的运行点:根据模型和评估维度的不同,低成本探索器保留了参考Hit@3指标的约78%-94%、F1指标的73%-92%,同时将智能体平均耗时降低41%-88%,token使用量降低84%-95%。最优运行点取决于下游对定位结果的使用方式:排名和覆盖率指标适用于可恢复的候选文件交接,而F1和精确匹配则适用于严格的文件准入场景。这些结果支持将代码仓库探索视为模块化编码智能体中可独立测量和预算的阶段,探索器的选择需由下游交接契约指导。
英文摘要
Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository files are likely to matter. We study whether this stage can be delegated to lower-cost models while retaining useful localization quality. Using our IssueLoc-Bench, we evaluate five explorer models under the same read-only interactive interface on 499 SWE-bench Verified-derived tasks and 500 tasks from 153 additional repositories. We measure early candidate discovery, top-three gold-file coverage, strict file-set recovery, agent time, and token usage, with paired instance-level uncertainty and repository-clustered sensitivity analysis. The highest-quality explorer leads across localization metrics, but substantially cheaper operating points emerge: depending on the model and evaluation arm, lower-cost explorers retain approximately 78-94% of the reference Hit@3 and 73-92% of its F1 while reducing mean agent time by 41-88% and token usage by 84-95%. The preferred operating point depends on how localization is consumed downstream: ranking and coverage metrics characterize recoverable candidate handoffs, whereas F1 and exact match characterize restrictive file gates. These results support treating repository exploration as an independently measurable and budgetable stage of modular coding agents, with explorer selection guided by the downstream handoff contract.