Large language models can generate runnable software artifacts, but their security remains difficult to evaluate end to end. This study examines that problem through a Detect--Repair--Verify (DRV) workflow, in which vulnerabilities are detected, repaired, and then rechecked with security and functional tests. It addresses four gaps in current evidence: the lack of test-grounded benchmarks for LLM-generated artifacts, limited evidence on pipeline-level effectiveness, unclear reliability of detection reports as repair guidance, and uncertain repair trustworthiness under verification.
To support this study, EduCollab is constructed as a multi-language, multi-granularity benchmark of runnable LLM-generated web applications in PHP, JavaScript, and Python. Each artifact is paired with executable functional and exploit test suites, and the benchmark spans project-, requirement-, and file-level settings. On this benchmark, the study compares unrepaired baselines, single-pass detect--repair, and bounded iterative DRV under comparable budget constraints. Outcomes are measured by secure-and-correct yield, and intermediate artifacts and iteration traces are analyzed to assess report actionability and repair failure modes.
The results show that bounded iterative DRV can improve secure-and-correct yield over single-pass repair, but the gains are uneven at the project level and become clearer at narrower repair scopes. Detection reports are often useful for downstream repair, but their reliability is inconsistent. Repair trustworthiness also depends strongly on repair scope. These findings highlight the need for test-grounded, end-to-end evaluation of LLM-based vulnerability management workflows.
ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming
最近的实证研究表明,GenProg的性能并不令人满意,特别是对于Java。在本文中,我们提出了ARJA,一个新的GP为基础的修复方法自动修复的Java程序。具体来说,我们提出了一种新的低粒度补丁表示,适当的搜索子空间的可能错误的位置,操作类型和潜在的修复成分,使GP探索搜索空间更有效。基于这种新的表示,我们制定自动程序修复作为一个多目标的搜索问题,并使用NSGA-II寻找更简单的修复。为了减少计算量和搜索空间,我们引入了一个测试过滤过程,可以加快适应度评估的GP和三种类型的规则,可以应用到避免不必要的操作的代码。此外,我们还提出了一种类型匹配策略,可以通过利用现有语句的语法模式来创建新的潜在修复成分。我们进行了一个大规模的实证评估ARJA及其变种的种子错误和现实世界的错误相比,几个国家的最先进的修复方法。我们的研究结果验证了ARJA中所采用的搜索机制的有效性和效率,也显示了其优于其他方法。特别是,与jGenProg(GenProg for Java的实现)相比,完全遵循冗余假设的ARJA版本可以为两倍以上的bug数量(从27到59)生成测试套件足够的补丁,并为近四倍的数量(从5到18)生成正确的补丁,在Defects 4J中考虑了224个真实世界的bug。此外,ARJA是能够正确地修复几个真正的多位置的错误,很难被修复的大多数现有的修复方法。
BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
State Key Laboratory of Networking and Switching Technology(网络与交换技术国家重点实验室);Beijing University of Posts and Telecommunications(北京邮电大学);University of Luxembourg(卢森堡大学)
Vul-R2: A Reasoning LLM for Automated Vulnerability Repair
Tencent Inc.(腾讯公司);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系)
From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
Shanghai Jiao Tong University(上海交通大学);University of California, Davis(加州大学戴维斯分校);East China Normal University(华东师范大学);University of Pennsylvania(宾夕法尼亚大学)
TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications(重庆邮电大学计算机科学与技术学院);Nanyang Technological University(南洋理工大学);Beihang University(北京航空航天大学);Southwest University(西南大学)