arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.04232cs.SEcs.CL
收藏

使用中间形式规范训练代码语言模型进行推理

Teaching Code LLMs to Reason with Intermediate Formal Specifications

FPT Software AI Center(FPT软件人工智能中心);University of Texas at Dallas(德克萨斯大学达拉斯分校);Hanoi Univ. of Science and Tech.(河内科学技术大学);Texas, USA(得克萨斯州,美国);Hanoi, Vietnam(河内,越南)

Minh Le-Anh, Cuong Chi Le, Tien N. Nguyen

AI总结:

研究可执行检查点规范生成,介绍SpecCoder框架从多种程序学习,能选正确规范拒错误执行。引入HumanExec基准测试,实验表明其提升了检查点规范质量及下游正确性推理和修复能力。

摘要:

与自然语言规范不同,可执行形式规范为代码验证等提供机器可检查约束,但编写费力,现有基于语言模型的方法有局限。本文研究可执行检查点规范生成,引入SpecCoder框架,介绍HumanExec基准测试,实验表明其显著提升了检查点规范质量及下游能力。

英文摘要:

Unlike natural-language specifications, executable formal specifications provide machine-checkable constraints for verifying, debugging, and repairing code. However, writing such specifications is labor-intensive, and existing LLM-based methods mainly infer whole-program pre/postconditions, missing the intermediate semantic commitments that programmers rely on when reasoning about an algorithm. Our study further shows that prompting current CodeLLMs often produces executable assertions that are syntactically invalid, trivial, or too weak to reject behavior-changing faults. In this paper, we study executable checkpoint specification generation, where assertions are inserted at meaningful internal program points to describe expected intermediate states. We introduce SpecCoder, a verification-guided CodeLLM training framework that learns from validated reference programs, behavior-changing mutants, and multi-turn specification-refinement traces. SpecCoder selects specifications that hold on correct executions while rejecting faulty executions, turning specifications from passive annotations into executable evidence. To evaluate this setting, we introduce HumanExec, a benchmark built from recent Codeforces competitive programming problems with test suites, reference solutions, and human buggy submissions, supporting three tasks: specification generation, program correctness checking, and program repair. Experiments on HumanExec show that SpecCoder substantially improves checkpoint-specification quality over base CodeLLMs. Across Qwen2.5-Coder models, SpecCoder improves inline-specification correctness by up to 55.8%, completeness by up to 358.1%, and executable assertion validity by up to 26.6%. These gains further translate to downstream correctness reasoning and repair, showing that executable checkpoints provide fine-grained evidence for reliable verification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑