In this paper, we propose an assertion-based approach to capture software evolution, through the notion of commit-relevant specification. A commit-relevant specification summarises the program properties that have changed as a consequence of a commit (understood as a specific software modification), via two sets of assertions, the delta-added assertions, properties that did not hold in the pre-commit version but hold on the post-commit, and the delta-removed assertions, those that were valid in the pre-commit, but no longer hold after the code change. We also present DeltaSpec, an approach that combines test generation and dynamic specification inference to automatically compute commit-relevant specifications from given commits. We evaluate DeltaSpec on two datasets that include a total of 57 commits (63 classes and 797 methods). We show that commit-relevant assertions can precisely describe the semantic deltas of code changes, providing a useful mechanism for validating the behavioural evolution of software. We also show that DeltaSpec can infer 88% of the manually written commit-relevant assertions expressible in the language supported by the tool. Moreover, our experiments demonstrate that DeltaSpec's inferred assertions are effective to detect regression faults. More precisely, we show that commit-relevant assertions can detect, on average, 78.3% of the artificially seeded faults that interact with the code changes. We also show that assertions in the delta are 58.3% more effective in detecting commit-relevant mutants than assertions outside the delta, and that it takes on average 169% fewer assertions when these are commit-relevant, compared to using general valid assertions, to achieve a same commit-relevant mutation score.
Improving Dynamic Specification Inference with LLM-Generated Counterexamples
University of Rio Cuarto（里奥夸尔托大学）；Complutense University of Madrid（马德里康普顿斯大学）；University of Buenos Aires（布宜诺斯艾利斯大学）；Luxembourg Institute of Science and Technology（卢森堡科学与技术研究所）；National Council for Scientific and Technical Research (CONICET)（国家科学研究和技术委员会）；Guangdong Technion-Israel Institute of Technology（广东以色列理工学院）
EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
Nanyang Technological University（南洋理工大学）；East China University of Science and Technology（东华大学）；Guangdong Ocean University（广东海洋大学）；City University of Hong Kong（香港城市大学）；Stanford University（斯坦福大学）
