Modern software systems complexity challenges efficient testing, as traditional machine learning (ML) struggles with large test suites. This research presents a hybrid framework integrating Quantum Annealing with ML to optimize test case prioritization in CI/CD pipelines. Leveraging quantum optimization, it achieves a 25 percent increase in defect detection efficiency and a 30 percent reduction in test execution time versus classical ML, validated on the Defects4J dataset. A simulated CI/CD environment demonstrates robustness across evolving codebases. Visualizations, including defect heatmaps and performance graphs, enhance interpretability. The framework addresses quantum hardware limits, CI/CD integration, and scalability for 2025s hybrid quantum-classical ecosystems, offering a transformative approach to software quality assurance.
Recent years have seen the successful application of deep learning to software engineering (SE). In particular, the development and use of pre-trained models of source code has enabled state-of-the-art results to be achieved on a wide variety of SE tasks. This paper provides an overview of this rapidly advancing field of research and reflects on future research directions.
Vector Institute for Artificial Intelligence（向量人工智能研究所）；University of Guelph（圭尔夫大学）
Although machine learning (ML) has been successful in automating various software engineering needs, software testing still remains a highly challenging topic. In this paper, we aim to improve the generative testing of software by directly augmenting the random number generator (RNG) with a deep reinforcement learning (RL) agent using an efficient, automatically extractable state representation of the software under test. Using the Cosmos SDK as the testbed, we show that the proposed DeepRNG framework provides a statistically significant improvement to the testing of the highly complex softwar…
Artificial Intelligence (AI) is making a significant impact in multiple areas like medical, military, industrial, domestic, law, arts as AI is capable to perform several roles such as managing smart factories, driving autonomous vehicles, creating accurate weather forecasts, detecting cancer and personal assistants, etc. Software testing is the process of putting the software to test for some abnormal behaviour of the software. Software testing is a tedious, laborious and most time-consuming process. Automation tools have been developed that help to automate some activities of the testing proc…
Machine learning has become prevalent across a wide variety of applications. Unfortunately, machine learning has also shown to be susceptible to deception, leading to errors, and even fatal failures. This circumstance calls into question the widespread use of machine learning, especially in safety-critical applications, unless we are able to assure its correctness and trustworthiness properties. Software verification and testing are established technique for assuring such properties, for example by detecting errors. However, software testing challenges for machine learning are vast and profuse…
Fraunhofer Institute for Experimental Software Engineering IESE（弗劳恩霍夫实验软件工程研究所）；Algorithm Accountability Lab, TU Kaiserslautern（凯泽斯劳滕工业大学算法问责实验室）
Analytical quality assurance, especially testing, is an integral part of software-intensive system development. With the increased usage of Artificial Intelligence (AI) and Machine Learning (ML) as part of such systems, this becomes more difficult as well-understood software testing approaches cannot be applied directly to the AI-enabled parts of the system. The required adaptation of classical testing approaches and the development of new concepts for AI would benefit from a deeper understanding and exchange between AI and software engineering experts. We see the different terminologies used …
Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature survey
Software testing is a prime factor in software industry. Besides knowing the importance of testing, only limited time is allocated for teaching it. It will be more efficient if testing is taught simultaneously with programming foundations. This integrated learning of testing techniques and programming allows the programmers to perform in a better way and this leads to the improvement of the performance of the industry progress. In this paper, a technique named ontology is introduced, it first defines the various testing process in hierarchy and define relationships among them, to share and reu…
Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding. Software testing is the process of testing the functionality and correctness of software by running it. Software testing is usually performed for one of two reasons: defect detection, and reliability estimation. The problem of applying software testing to defect detection is that software can only suggest the presence of flaws, not their absence (unless the testing is exhaustive). The problem of applying software testing to reliability estimation is that …
Federal Institute of the Southeast of Minas Gerais (IF Sudeste MG)（米纳斯吉拉斯东南联邦学院）；Federal Institute of Minas Gerais (IFMG)（米纳斯吉拉斯联邦学院）；Federal University of Minas Gerais (UFMG)（米纳斯吉拉斯联邦大学）
Deep Learning (DL) is being used nowadays in many traditional Software Engineering (SE) problems and tasks. However, since the renaissance of DL techniques is still very recent, we lack works that summarize and condense the most recent and relevant research conducted at the intersection of DL and SE. Therefore, in this paper, we describe the first results of a mapping study covering 81 papers about DL & SE. Our results confirm that DL is gaining momentum among SE researchers over the years and that the top-3 research problems tackled by the analyzed papers are documentation, defect prediction,…
Software testing is an expensive process, which is vital in the industry. Construction of the test-data in software testing requires the major cost and to decide which method to use in order to generate the test data is important. This paper discusses the efficiency of search-based algorithms (preferably genetic algorithm) versus random testing, in soft- ware test-data generation. This study differs from all previous studies due to sample programs (SUTs) which are used. Since we want to in- crease the complexity of SUTs gradually, and the program generation is automatic as well, Grammatical Ev…
TestLab: An Intelligent Automated Software Testing Framework
Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)（高级创新与发展智能工程与计算研究组（GECAD））；Porto School of Engineering (ISEP)（波尔图工程学院（ISEP））
本文提出智能自动化软件测试框架 TestLab，利用 AI 集成三类模块，从不同视角发现漏洞并基于源代码分析自动生成测试用例，以支持多层级持续测试。
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
This paper defines software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates, focusing on causality in discriminatory behavior. Evidence of software discrimination has been found in modern software systems that recommend criminal sentences, grant access to financial products, and determine who is allowed to participate in promotions. Our approach, Themis, generates efficient test suites to measure discrimination. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not req…
Context: Scientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract publications due to errors caused by software faults. Systematic testing can identify such faults in code. Objective: This study aims to identify specific challenges, proposed solutions, and unsolved problems faced when testing scientific software. Method: We conducted a systematic literature survey to identify and analyze relevant literature. We identifie…
Volvo Group（沃尔沃集团）；Chalmers University of Technology（查尔姆斯理工大学）
Automotive software testing continues to rely largely upon expensive field tests to ensure quality because alternatives like simulation-based testing are relatively immature. As a step towards lowering reliance on field tests, we present SilGAN, a deep generative model that eases specification, stimulus generation, and automation of automotive software-in-the-loop testing. The model is trained using data recorded from vehicles in the field. Upon training, the model uses a concise specification for a driving scenario to generate realistic vehicle state transitions that can occur during such a s…
Background: Research software plays an important role in solving real-life problems, empowering scientific innovations, and handling emergency situations. Therefore, the correctness and trustworthiness of research software are of absolute importance. Software testing is an important activity for identifying problematic code and helping to produce high-quality software. However, testing of research software is difficult due to the complexity of the underlying science, relatively unknown results from scientific algorithms, and the culture of the research software community. Aims: The goal of thi…
Cloud computing is the next stage of the internet evolution. It relies on sharing of resources to achieve coherence on a network. It is emerged as new computing standard that impacts several different research fields, including software testing. There are various software techniques used for testing application. It not only changes the way of obtaining computing resources but also changes the way of managing and delivering computing services, technologies and solutions, meanwhile it causes new issues, challenges and needs in software testing. Software testing in cloud can reduce the need for h…
Context: With the rising complexity and scale of software systems, there is an ever-increasing demand for sophisticated and cost-effective software testing. To meet such a demand, there is a need for a highly-skilled software testing work-force (test engineers) in the industry. To address that need, many university educators worldwide have included software-testing education in their software engineering (SE) or computer science (CS) programs. Objective: Our objective in this paper is to summarize the body of experience and knowledge in the area of software-testing education to benefit the rea…
Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning
Advanced Institute of So-Go-Chi (Convergence Knowledge) Informatics（融合知识研究院）；Tohoku University（东北大学）；Japan Advanced Institute of Science and Technology（日本先进科学研究院）；Faculty of Information and Communication Technology（信息与通信技术学院）；Mahidol University（玛希敦大学）
Today's software systems like cyber-physical production systems or big data systems have to process large volumes and diverse types of data which heavily influences the quality of these so-called data-intensive systems. However, traditional software testing approaches rather focus on functional behavior than on data aspects. Therefore, the role of data in testing has to be rethought and specific testing approaches for data-intensive software systems are required. Thus, the aim of this chapter is to contribute to this area by (1) providing basic terminology and background on data-intensive soft…
Fuzzy Inference System for Test Case Prioritization in Software Testing
School of Information Technology and Engineering（信息技术与工程学院）；Kazakh-British Technical University（哈萨克斯坦-英国技术大学）；Faculty of Information Technology（信息技术学院）
As software systems are becoming more pervasive, they are also becoming more susceptible to failures, resulting in potentially lethal combinations. Software testing is critical to preventing software failures but is, arguably, the least understood part of the software life cycle and the toughest to perform correctly. Adequate research has been carried out in both the process and technology dimensions of testing, but not in the human dimensions. This paper attempts to fill in the gap by exploring the human dimension, i.e., trying to understand the motivation of software professionals to take up…
The overall aim of the software industry is to ensure delivery of high quality software to the end user. To ensure high quality software, it is required to test software. Testing ensures that software meets user specifications and requirements. However, the field of software testing has a number of underlying issues like effective generation of test cases, prioritisation of test cases etc which need to be tackled. These issues demand on effort, time and cost of the testing. Different techniques and methodologies have been proposed for taking care of these issues. Use of evolutionary algorithms…
This study attempts to understand motivators and de-motivators that influence the decisions of software students to take up and sustain software testing careers across three different Asian countries, i.e., China, India, and Malaysia. The re-search question can be framed as How many software students across different Asian geographies are keen to take up testing careers, and what are the reasons for their choices? Towards an answer, we developed a cross-sectional but simple survey-based instrument. In this work, we investigated how software students perceived the software testing role. The res…
This note concerns a search for publications in which the pragmatic concept of a test as conducted in the practice of software testing is formalized, a theory about software testing based on such a formalization is presented or it is demonstrated on the basis of such a theory that there are solid grounds to test software in cases where in principle other forms of analysis could be used. This note reports on the way in which the search has been carried out and the main outcomes of the search. The message of the note is that the fundamentals of software testing are not yet complete in some respe…
The paper presents the experience of the authors in model based testing of safety critical real-time control logic software. It describes specifics of the corresponding industrial settings and discusses technical details of usage of UniTESK model based testing technology in these settings. Finally, we discuss possible future directions of safety critical software development processes and a place of model based testing techniques in it.
Ongoing progress in computational intelligence (CI) has led to an increased desire to apply CI techniques for the purpose of improving software engineering processes, particularly software testing. Existing state-of-the-art automated software testing techniques focus on utilising search algorithms to discover input values that achieve high execution path coverage. These algorithms are trained on the same code that they intend to test, requiring instrumentation and lengthy search times to test each software component. This paper outlines a novel genetic programming framework, where the evolved …
Software testing is an important and valuable part of the software development life cycle. Due to time, cost and other circumstances, exhaustive testing is not feasible that's why there is a need to automate the software testing process. Testing effectiveness can be achieved by the State Transition Testing (STT) which is commonly used in real time, embedded and web-based type of software systems. Aim of the current paper is to present an algorithm by applying an ant colony optimization technique, for generation of optimal and minimal test sequences for behavior specification of software. Prese…
Software testing is vital to improve software quality. However, software tester role is stigmatized, partly due to misperception and partly due to the treatment of the testing process within the software industry. The present study analyses this situation aiming to explore what might inhibit an individual from taking up a software testing career. In order to investigate this issue, we surveyed 132 senior students pursuing degrees in information systems, information and communication technology, computer science, computer engineering, software engineering, and other closely-related disciplines …
This paper attempts to understand motivators and de-motivators that influence the decisions of software professionals to take up and sustain software testing careers across four different countries, i.e. Canada, China, Cuba, and India. The research question can be framed as "How many software professionals across different geographies are keen to take up testing careers, and what are the reasons for their choices?" Towards that, we developed a cross-sectional but simple survey-based instrument. In this study we investigated how software testers perceived and valued what they do and their envir…
Software testing uses wide range of different tools to enhance the complicated process of defining quality of the system under test. Formal Concept Analysis (FCA) provides us with algorithms of deriving formal ontology from a set of objects and their attributes. With the use of FCA we can considerably improve the efficiency of test case derivation. Moreover, an FCA-based machine learning system supports the analysis of regression testing results.
Software testing is one of the crucial supporting processes of software life cycle. Unfortunately for the software industry, the role is stigmatized, partly due to misperception and partly due to treatment of the role in the software industry. The present study aims to analyse this situation to explore what inhibit an individual from taking up a software testing career. In order to investigate this issue, we surveyed 82 senior students taking a degree in information technology, information and communication technology, and computer science at two Malaysian universities. The subjects were asked…
Context: While many companies conduct their software testing activities in-house, many other companies outsource their software testing needs to other firms who act as software testing service providers. As a result, Testing as a Service (TaaS) has emerged as a strong service industry in the last several decades. In the context of software testing services, there could be various challenges (e.g., during the planning and service delivery phases) and, as a result, the quality of testing services is not always as expected. Objective: It is important, for both providers and also customers of test…
Context: Software testing plays an essential role in product quality improvement. For this reason, several software testing models have been developed to support organizations. However, adoption of testing process models inside organizations is still sporadic, with a need for more evidence about reported experiences. Aim: Our goal is to identify results gathered from the application of software testing models in organizational contexts. We focus on characteristics such as the context of use, practices applied in different testing process phases, and reported benefits & drawbacks. Method: We pe…
Software engineering educators are continually challenged by rapidly evolving concepts, technologies, and industry demands. Due to the omnipresence of software in a digitalized society, higher education institutions (HEIs) have to educate the students such that they learn how to learn, and that they are equipped with a profound basic knowledge and with latest knowledge about modern software and system development. Since industry demands change constantly, HEIs are challenged in meeting such current and future demands in a timely manner. This paper analyzes the current state of practice in soft…
Research software is often developed by individual researchers or small teams in parallel to their research work. The more people and research projects rely on the software in question, the more important it is that software updates implement new features correctly and do not introduce regressions. Thus, developers of research software must balance their limited resources between implementing new features and thoroughly testing any code changes. We present the processes we use for developing the distributed integration framework RCE at DLR. These processes aim to strike a balance between autom…
