arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.02407cs.SEcs.LG
收藏

用合成缺陷挑战缺陷预测与修复模型

Challenging Bug Prediction and Repair Models with Synthetic Bugs

University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stanford University（斯坦福大学）

Ali Reza Ibrahimzada, Yang Chen, Ryan Rong, Reyhaneh Jabbarvand

备注：
Published in SCAM 2025
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出BugFarm，利用LLM在注意力最少位置修改代码，生成难以检测和修复的复杂缺陷，评估显示其优于现有方法。

摘要：

缺陷在软件工程中至关重要；过去几十年中，许多研究被提出用于检测、定位和修复软件系统中的缺陷。此类技术的有效性评估需要复杂的缺陷，即那些难以通过测试检测且难以通过调试修复的缺陷。从经典软件工程的角度来看，难以修复的缺陷与正确代码在多个位置不同，这使得定位和修复变得困难。另一方面，难以检测的缺陷在特定的测试输入和可达性条件下显现。这两个目标，即生成难以检测和难以修复的缺陷，在很大程度上是一致的；缺陷生成技术可以修改多个语句，使其仅在特定输入集下被覆盖。然而，对于基于学习的技术，这两个目标是冲突的：缺陷应与训练数据中的正确代码具有相似的代码表示，以挑战缺陷预测模型区分它们。难以修复的缺陷定义保持不变，但有一个注意事项：缺陷与原始代码差异越大，它们的表示就越远，越容易被检测。我们提出了BugFarm，将任意代码转换为多个复杂缺陷。BugFarm利用大型语言模型在多个位置修改代码（难以修复）。为确保多次修改不会显著改变代码表示，BugFarm分析底层模型的注意力，并指示LLMs仅修改注意力最少的位置（难以检测）。我们对由BUGFARM和两种替代方法生成的超过190万个突变体中的43.5万多个缺陷进行的全面评估表明，我们在生成难以被基于学习的缺陷预测方法检测和难以被最先进的基于学习的程序修复技术修复的缺陷方面具有优越性。

英文摘要：

Bugs are essential in software engineering; many research studies in the past decades have been proposed to detect, localize, and repair bugs in software systems. Effectiveness evaluation of such techniques requires complex bugs, i.e., those that are hard to detect through testing and hard to repair through debugging. From the classic software engineering point of view, a hard-to-repair bug differs from the correct code in multiple locations, making it hard to localize and repair. Hard-to-detect bugs, on the other hand, manifest themselves under specific test inputs and reachability conditions. These two objectives, i.e., generating hard-to-detect and hard-to-repair bugs, are mostly aligned; a bug generation technique can change multiple statements to be covered only under a specific set of inputs. However, these two objectives are conflicting for learning-based techniques: A bug should have a similar code representation to the correct code in the training data to challenge a bug prediction model to distinguish them. The hard-to-repair bug definition remains the same but with a caveat: the more a bug differs from the original code, the more distant their representations are and easier to be detected. We propose BugFarm, to transform arbitrary code into multiple complex bugs. BugFarm leverages LLMs to mutate code in multiple locations (hard-to-repair). To ensure that multiple modifications do not notably change the code representation, BugFarm analyzes the attention of the underlying model and instructs LLMs to only change the least attended locations (hard-to-detect). Our comprehensive evaluation of 435k+ bugs from over 1.9M mutants generated by BUGFARM and two alternative approaches demonstrates our superiority in generating bugs that are hard to detect by learning-based bug prediction approaches and hard-to-repair by state-of-the-art learning-based program repair technique.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑