Test case generation is an important activity, yet a time-consuming and laborious task. Recently, AthenaTest -- a deep learning approach for generating unit test cases -- is proposed. However, AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. In this paper, we propose A3Test, a DL-based test case generation approach that is augmented by assertion knowledge with a mechanism to verify naming consistency and test signatures. A3Test leverages the domain adaptation principles where the goal is to adapt the existing knowledge from an assertion generation task to the test case generation task. We also introduce a verification approach to verify naming consistency and test signatures. Through an evaluation of 5,278 focal methods from the Defects4j dataset, we find that our A3Test (1) achieves 147% more correct test cases and 15% more method coverage, with a lower number of generated test cases than AthenaTest; (2) still outperforms the existing pre-trained models for the test case generation task; (3) contributes substantially to performance improvement via our own proposed assertion pre-training and the verification components; (4) is 97.2% much faster while being more accurate than AthenaTest.
This demo presents the implementation and usage details of GASSERT, the first tool to automatically improve assertion oracles. Assertion oracles are executable boolean expressions placed inside the program that should pass (return true) for all correct executions and fail (return false) for all incorrect executions. Because designing perfect assertion oracles is difficult, assertions are prone to both false positives (the assertion fails but should pass) and false negatives (the assertion passes but should fail). Given a Java method containing an assertion oracle to improve, GASSERT returns an…
Converting source or unit test code to English has been shown to improve the maintainability, understandability, and analysis of software and tests. Code summarizers identify important statements in the source/tests and convert them to easily understood English sentences using static analysis and NLP techniques. However, current test summarization approaches handle only a subset of the variation and customization allowed in the JUnit assert API (a critical component of test cases) which may affect the accuracy of conversions. In this paper, we present our work towards improving JUnit test summ…
Functional validation is necessary to detect any errors during quantum computation. There are promising avenues to debug quantum circuits using runtime assertions. However, the existing approaches rely on the expertise of the verification engineers to manually design and insert the assertions in suitable locations. In this paper, we propose automated generation and placement of quantum assertions based on static analysis and random sampling of quantum circuits. Specifically, this paper makes two important contributions. We automatically uncover special properties of a quantum circuit, such as …
AutoAssert 1: A LoRA Fine-Tuned LLM Model for Efficient Automated Assertion Generation
Institute of Computing Technology（计算技术研究所）；Chinese Academy of Sciences（中国科学院）；International College（国际学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；University of Chinese Academy of Sciences（中国科学院大学）
Test smells can reduce the developers' ability to interact with the test code. Refactoring test code offers a safe strategy to handle test smells. However, the manual refactoring activity is not a trivial process, and it is often tedious and error-prone. This study aims to evaluate RAIDE, a tool for automatic identification and refactoring of test smells. We present an empirical assessment of RAIDE, in which we analyzed its capability at refactoring Assertion Roulette and Duplicate Assert test smells and compared the results against both manual refactoring and a state-of-the-art approach. The …
Automated Vehicles (AVs) are rapidly maturing in the transportation domain. However, the complexity of the AV design problem is such that no single technique is sufficient to provide adequate validation of key properties such as safety, reliability or trustworthiness. In this vision paper, a combination of a spatial traffic logic and agent-based verification methods with a validation method that uses assertion checking of simulations is proposed. We sketch how to integrate the respective approaches within a methodological framework called Corroborative Verification and Validation (V&V).The Cor…
Various mature automated test generation tools exist for statically typed programming languages such as Java. Automatically generating unit tests for dynamically typed programming languages such as Python, however, is substantially more difficult due to the dynamic nature of these languages as well as the lack of type information. Our Pynguin framework provides automated unit test generation for Python. In this paper, we extend our previous work on Pynguin to support more aspects of the Python language, and by studying a larger variety of well-established state of the art test-generation algor…
Rotten green tests are passing tests which have, at least, one assertion not executed. They give developers a false confidence. In this paper, we present, RTj, a framework that analyzes test cases from Java projects with the goal of detecting and refactoring rotten test cases. RTj automatically discovered 427 rotten tests from 26 open-source Java projects hosted on GitHub. Using RTj, developers have an automated recommendation of the tests that need to be modified for improving the quality of the applications under test.
Retrieval-Augmented Instruction Tuning for Automated Process Engineering Calculations : A Tool-Chaining Problem-Solving Framework with Attributable Reflection
Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verification
University of California San Diego（加利福尼亚大学圣迭戈分校）；INESC-ID（INESC-ID（里斯本计算机与系统工程研究所））；IST, University of Lisbon（里斯本大学高等技术学院）；Imperial College London（伦敦帝国学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Rango 是一个面向 Coq 的全自动证明合成工具，通过在每步证明中利用检索增强技术自适应地引入相关前提和相似证明，在自建基准上比 SOTA 工具 Tactician 多证明 29% 的定理。
AutoINV: Automated Invariant Generation Framework for Formal Verification on High-Level Synthesis Designs
The Hong Kong University of Science and Technology（香港科技大学）；Indian Institute of Technology Goa（印度理工学院果阿分校）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
