arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.00897cs.SE
收藏

检测修复验证以确保LLM生成的代码安全:一个多语言实证研究

Detect Repair Verify for Securing LLM Generated Code: A Multi-Language Empirical Study

Concordia University(康考迪亚大学)

Cheng Cheng

AI总结:

本文通过多语言实证研究,提出了一种用于确保LLM生成代码安全的检测-修复-验证流程,并构建了新的基准数据集以评估不同修复方法的有效性。

摘要:

大型语言模型越来越多地用于生成可运行的软件。在实践中,安全通常是通过检测-修复-验证(DRV)循环来解决的,该循环检测问题、应用修复并验证结果。这项工作研究了此类工作流程用于项目级 artifacts,并解决了四个缺口:L1,缺乏具有可执行功能和安全测试的项目级基准;L2,有限的证据证明在仅研究检测或修复之外的流程级有效性;L3,检测报告的可靠性不明确作为修复指导;L4,验证下修复的可信度和副作用不确定。引入了一个新的基准数据集,由可运行的Web应用程序项目配对功能测试和目标安全测试组成,并支持在项目、需求和功能级别三个提示粒度。评估比较了仅生成、单次 DRV 和有界迭代 DRV 变体在可比预算限制下的表现。结果通过使用基于测试的验证测量安全和正确产出,并分析中间制品以评估报告的可操作性和修复后失败模式,如回归、语义漂移和新引入的安全问题。

英文摘要:

Large language models are increasingly used to produce runnable software. In practice, security is often addressed through a Detect--Repair--Verify (DRV) loop that detects issues, applies fixes, and verifies the result. This work studies such a workflow for project-level artifacts and addresses four gaps: L1, the lack of project-level benchmarks with executable function and security tests; L2, limited evidence on pipeline-level effectiveness beyond studying detection or repair alone; L3, unclear reliability of detection reports as repair guidance; and L4, uncertain repair trustworthiness and side effects under verification. A new benchmark dataset\footnote{https://github.com/Hahappyppy2024/EmpricalVDR} is introduced, consisting of runnable web-application projects paired with functional tests and targeted security tests, and supporting three prompt granularities at the project, requirement, and function level. The evaluation compares generation-only, single-pass DRV, and bounded iterative DRV variants under comparable budget constraints. Outcomes are measured by secure and correct yield using test-grounded verification, and intermediate artifacts are analyzed to assess report actionability and post-repair failure modes such as regressions, semantic drift, and newly introduced security issues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑