arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21697cs.SE
收藏

EditFlow:通过开发者流程重建实现代码编辑推荐系统的基准测试与优化

EditFlow: Benchmarking and Optimizing Code Edit Recommendation Systems via Reconstruction of Developer Flows

Shanghai Jiao Tong University(上海交通大学);National University of Singapore(新加坡国立大学);Bytedance Network Technology(字节跳动网络科技)

Chenyan Liu, Yun Lin, Jiaxin Chang, Jiawei Liu, Binhang Qi, Bo Jiang, Zhiyong Huang, Jin Song Dong

AI总结:

本文提出EditFlow,通过重建开发者编辑流程,解决代码编辑推荐系统在数据收集、基准测试和统一优化中的关键挑战,提升开发者生产力。

摘要:

大型语言模型(LLMs)在代码编辑方面已取得显著进展,但近期实证研究揭示了技术准确性和开发者生产力之间的根本性脱节。尽管其在基准测试中表现优异,但开发者在使用AI辅助时完成任务的速度仍慢19%,超过68.81%的推荐会打断他们的思维流程。这种不匹配源于静态提交快照的使用,缺乏时间信息,导致模型优化终点结果而非增量、情境敏感的步骤,这与开发者自然的推理过程不一致。为弥合这一差距,我们提出了EditFlow,通过重建开发者编辑流程来基准测试和优化后续代码编辑推荐系统。EditFlow解决了三个关键挑战:首先,收集反映开发者流程的编辑顺序数据本质上困难:手动注释引入了不可接受的开销,而开发日志只能捕捉单一轨迹而非所有可能的编辑流程。其次,将推荐性能基准测试与开发者正在进行的编辑流程相结合,需要类似数字双胞胎的模拟,能够忠实模拟编辑过程。第三,现有异构系统在规模和架构上差异巨大,给开发统一优化策略带来挑战,该策略需要使所有模型具备思维流程意识,无论设计或能力如何。

英文摘要:

Large language models (LLMs) for code editing have achieved remarkable progress, yet recent empirical studies reveal a fundamental disconnect between technical accuracy and developer productivity. Despite their strong benchmark performance, developers complete tasks 19% slower when using AI assistance, with over 68.81% of recommendations disrupting their mental flow. This misalignment stems from the use of static commit snapshots that lack temporal information, causing models to optimize for end results rather than the incremental, context-sensitive steps that align with developers' natural reasoning process. To bridge this gap, we present EditFlow, which benchmarks and optimizes subsequent code edit recommendation systems through the reconstruction of developer editing flows. EditFlow addresses three key challenges. First, collecting edit-order data that reflects developers' flow is inherently difficult: manual annotation introduces prohibitive overhead, while development logs capture only single trajectories instead of all plausible editing flows. Second, benchmarking recommendation performance against developers' ongoing editing flow requires a digital-twin-like simulation that can faithfully simulate the editing process. Third, existing heterogeneous systems vary drastically in scale and architecture, posing challenges for developing a unified optimization strategy that endows all models with mental-flow awareness regardless of design or capability. ......

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑