PatchGuru: 通过大语言模型从自然语言制品推断补丁Oracle
PatchGuru: Patch Oracle Inference from Natural Language Artifacts
AI总结:
PatchGuru通过大语言模型从自然语言制品推断补丁规范,提高补丁验证的准确性与效率。
摘要:
随着软件系统的演变,补丁可能会无意中改变程序行为。由于回归测试不完整以及非可执行的自然语言(NL)描述了补丁意图,验证补丁的语义很困难。我们提出了PatchGuru,这是首个自动技术,从真实世界中的拉取请求(PRs)中推断出可执行的补丁规范。给定一个PR,PatchGuru使用大语言模型(LLMs)从自然语言制品中提取开发者意图,并合成补丁Oracle:在比较程序中以运行时断言形式表达的下界且实用的规范,这些程序集成了预补丁和后补丁版本。补丁Oracle专注于补丁相关的行为,使自动化验证成为可能,并支持跨版本属性。PatchGuru通过比较预补丁和后补丁行为迭代地细化推断的Oracle,识别违规情况,通过自我审查过滤不一致,并生成bug报告。我们在四个广泛使用的开源Python项目中评估了PatchGuru,共处理了400个最近的PRs。PatchGuru报告了39个警告,精确度为0.62,产生24个确认的真正阳性结果,包括12个此前未知的bug,其中11个随后由开发者修复。与最先进的技术Testora相比,PatchGuru检测到17个额外的bug(24 vs. 7),同时将精确度从0.32提升到0.62。PatchGuru的平均成本为每个PR 8.9分钟和0.07美元。这些结果表明,PatchGuru通过提供可执行文档和自动验证补丁意图来补充代码审查和回归测试。
英文摘要:
As software systems evolve, patches may unintentionally alter program behavior. Validating patches against their intended semantics is difficult due to incomplete regression tests and informal, non-executable natural language (NL) descriptions of patch intent. We present PatchGuru, the first automated technique that infers executable patch specifications from real-world pull requests (PRs). Given a PR, PatchGuru uses large language models (LLMs) to extract developer intent from NL artifacts and synthesizes patch oracles: under-approximate yet practical specifications expressed as runtime assertions in comparison programs that integrate pre- and post-patch versions. Patch oracles focus on patch-relevant behaviors, enable automated validation, and support cross-version properties. PatchGuru iteratively refines inferred oracles by comparing pre- and post-patch behaviors, identifies violations, filters inconsistencies via self-review, and generates bug reports. We evaluate PatchGuru on 400 recent PRs from four widely used open-source Python projects. PatchGuru reports 39 warnings with a precision of 0.62, yielding 24 confirmed true positives, including 12 previously unknown bugs, 11 of which were subsequently fixed by developers. Compared to the state-of-the-art technique Testora, PatchGuru detects 17 more bugs (24 vs. 7) while improving precision from 0.32 to 0.62. PatchGuru incurs an average cost of 8.9 minutes and USD 0.07 per PR. These results suggest that PatchGuru complements code review and regression testing by providing executable documentation and automated validation of patch intent.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。