利用代码智能体进行自动软件验证
Harnessing Code Agents for Automatic Software Verification
AI总结:
研究利用大语言模型代码智能体进行自动软件验证,将引理交给智能体自由选择方法并置于验证框架,该方法简单有效,能完全覆盖目标引理,在多维度评估中表现出色,可自动为软件开发编写证明。
摘要:
形式验证能为软件正确性提供最强保障,但难以扩展,如Coq等交互式定理证明器所需的证明需大量专家精力。大语言模型有望自动生成这些证明,然而现有方法将固定的、人为设计的证明策略植入系统并限制模型遵循,仍只能证明部分目标定理。我们表明施加此类策略既不必要也有局限性。将整个引理交给通用的大语言模型代码智能体(如Claude Code)自由选择方法,并将其置于验证框架中,更简单有效,能实现完全覆盖,即每个目标引理都能被证明,无需失败且无需Coq专家干预。智能体在框架的反馈和严格约束下编写证明,使每个证明合理(仅当证明器内核关闭时才被接受)、完整(无未证明或被默默放弃的义务)且终止(无发散策略)。我们从三个维度评估此框架加代码智能体。一是核心逻辑:在Iris(用于并发和内存操作程序的最先进分离逻辑)上,Aria自动证明了四个核心模块的所有4257个引理以及基于它构建的验证Rust标准库的217个引理。二是与先前大语言模型证明器比较:在reglang上,先前证明器 barely one in eight,而Aria证明了所有318个。三是通用性:在iris-lean(Iris未完成的Lean 4端口)上,它证明了72个尚未移植的引理,表明该方法不特定于Coq。一个最先进的模型(Claude Opus 4.7)可以为经过验证的软件开发完全自动地编写证明。
英文摘要:
Formal verification offers the strongest guarantee of software correctness, but it does not scale: the proofs demanded by interactive theorem provers such as Coq require enormous expert effort. Large language models (LLMs) promise to generate these proofs automatically, yet existing approaches wire a fixed, human-designed proof strategy into the system and constrain the model to follow it (retrieving premises and predicting tactics one step at a time, or splitting goals by divide-and-conquer), and still prove only a fraction of their target theorems. We show that imposing such a strategy is unnecessary and limiting. Handing the whole lemma to a general LLM code agent (for example, Claude Code), free to choose its own approach, and wrapping it in a verification harness is both simpler and more effective, achieving full coverage: every targeted lemma proved, with no failures and no Coq expert intervention. The agent writes the proofs under feedback and hard constraints from the harness that keep each one sound (accepted only when the prover's kernel closes it), complete (no obligation left unproved or silently dropped), and terminating (no divergent tactics). We evaluate this harness plus code agent along three dimensions. (1) Core logic: on Iris, the state-of-the-art separation logic for concurrent and memory-manipulating programs, Aria proves all 4,257 lemmas of the four core modules and the 217 lemmas verifying Rust's standard libraries built on it, fully automatically. (2) Comparison with prior LLM provers: on reglang, where prior provers manage barely one in eight, Aria proves all 318. (3) Generality: on iris-lean, the unfinished Lean 4 port of Iris, it proves 72 not-yet-ported lemmas, showing the approach is not specific to Coq. A state-of-the-art model (Claude Opus 4.7) can write proofs for verified software development fully and automatically.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。