Context: Software testability is the degree to which a software system or a unit under test supports its own testing. To predict and improve software testability, a large number of techniques and metrics have been proposed by both practitioners and researchers in the last several decades. Reviewing and getting an overview of the entire state-of-the-art and state-of-the-practice in this area is often challenging for a practitioner or a new researcher. Objective: Our objective is to summarize the body of knowledge in this area and to benefit the readers (both practitioners and researchers) in preparing, measuring and improving software testability. Method: To address the above need, the authors conducted a survey in the form of a systematic literature mapping (classification) to find out what we as a community know about this topic. After compiling an initial pool of 303 papers, and applying a set of inclusion/exclusion criteria, our final pool included 208 papers. Results: The area of software testability has been comprehensively studied by researchers and practitioners. Approaches for measurement of testability and improvement of testability are the most-frequently addressed in the papers. The two most often mentioned factors affecting testability are observability and controllability. Common ways to improve testability are testability transformation, improving observability, adding assertions, and improving controllability. Conclusion: This paper serves for both researchers and practitioners as an "index" to the vast body of knowledge in the area of testability. The results could help practitioners measure and improve software testability in their projects.
Software measurement is an essential management tool to develop robust and maintainable software systems. Software metrics can be used to control the inherent complexities in software design. To guarantee that the components of the software are inevitably testable, the testability attribute is used, which is a sub-characteristics of the software's maintabilility as well as quality assurance. This study investigates the relationship between static code and test metrics and testability and test cases effectiveness. The study answers three formulated research questions. The results of the analysi…
Testability is the probability whether tests will detect a fault, given that a fault in the program exists. How efficiently the faults will be uncovered depends upon the testability of the software. Various researchers have proposed qualitative and quantitative techniques to improve and measure the testability of software. In literature, a plethora of reliability growth models have been used to assess and measure the quantitative quality assessment of software during testing and operational phase. The knowledge about failure distribution and their complexity can improve the testability of soft…
Evaluating Software testability can assist software managers in optimizing testing budgets and identifying opportunities for refactoring. In this paper, we abandon the traditional approach of pursuing testability measurements based on the correlation between software metrics and test characteristics observed on past projects, e.g., the size, the organization or the code coverage of the test cases. We propose a radically new approach that exploits automatic test generation and mutation analysis to quantify the amount of evidence about the relative hardness of identifying effective test cases. W…
Measuring testability early in the development life cycle especially at design phase is a criterion of crucial importance to software designers, developers, quality controllers and practitioners. However, most of the mechanism available for testability measurement may be used in the later phases of development life cycle. Early estimation of testability, absolutely at design phase helps designers to improve their designs before the coding starts. Practitioners regularly advocate that testability should be planned early in design phase. Testability measurement early in design phase is greatly e…
The pivotal role of testing in high-quality software production has driven a significant effort in evaluating and assessing testing practices. We explore the state of testing in a large industrial project over an extended period. We study the interplay between bugs in the project and its test cases, and interview developers and stakeholders to uncover reasons underpinning our observations. We realized that testing is not well adopted, and that testability (ie, ease of testing) is low. We found that developers tended to abandon writing tests when they assessed the effort to be high. Frequent ch…
Most businesses rely on a significant stack of software to perform their daily operations. This software is business-critical as defects in this software have major impacts on revenue and customer satisfaction. The primary means for verification of this software is testing. We conducted an extensive analysis of Java software packages to evaluate their unit-testability. The results show that code in software repositories is typically split into portions of very trivial code, non-trivial code that is unit-testable, and code that cannot be unit-tested easily. This brings up interesting considerat…
Unlike most other software quality attributes, testability cannot be evaluated solely based on the characteristics of the source code. The effectiveness of the test suite and the budget assigned to the test highly impact the testability of the code under test. The size of a test suite determines the test effort and cost, while the coverage measure indicates the test effectiveness. Therefore, testability can be measured based on the coverage and number of test cases provided by a test suite, considering the test budget. This paper offers a new equation to estimate testability regarding the size…
Declarative styles such as functional programming (FP) are rapidly gaining ground on their imperative cousins, including procedural and object-oriented programming. The shift is subtle because it is happening within the context of multiparadigm programming languages such as JavaScript. FP is better suited to modern processes like test-driven development (TDD), and architectures like massively parallel, cloud-based computing. This article describes the technical details that characterize the shift from imperative to FP and implications for software quality management, particularly reuse and tes…
Software developers and maintainers need to read and understand source programs and other software artifacts. The increase in size and complexity of software drastically affects several quality attributes, especially understandability and maintainability. False interpretation often leads to ambiguities, misunderstanding and hence to faulty development results. Despite the fact that software understandability is vital and one of the most significant components of the software development process, it is poorly managed. This is mainly due to the lack of its proper management and control. The pape…
Recent years, deep learning is increasingly prevalent in the field of Software Engineering (SE). However, many open issues still remain to be investigated. How do researchers integrate deep learning into SE problems? Which SE phases are facilitated by deep learning? Do practitioners benefit from deep learning? The answers help practitioners and researchers develop practical deep learning models for SE tasks. To answer these questions, we conduct a bibliography analysis on 98 research papers in SE that use deep learning techniques. We find that 41 SE tasks in all SE phases have been facilitated…
Context: Understanding the types of software engineering practices and techniques used in the industry is important. There is a wide spectrum in terms of the types and maturity of software engineering practices conducted in each software team and company. To characterize the type of software engineering practices conducted in software firms, a variety of surveys have been conducted in different countries and regions. Turkey has a vibrant software industry and it is important to characterize and understand the state of software engineering practices in this industry. Objective: Our objective is…
Software reliability analysis is performed at various stages during the process of engineering software as an attempt to evaluate if the software reliability requirements have been (or might be) met. In this report, I present a summary of some fundamental black-box and white-box software reliability models. I also present some general shortcomings of these models and suggest avenues for further research.
Software product lines have recently been presented as one of the best promising improvements for the efficient software development. Different research works contribute supportive parameters and negotiations regarding the problems of producing a perfect software scheme. Traditional approaches or recycling software are not effective to solve the problems concerning software competence. Since fast developments with software engineering in the past few years, studies show that some approaches are getting extensive attention in both industries and universities. This method is categorized as the s…
The overall aim of the software industry is to ensure delivery of high quality software to the end user. To ensure high quality software, it is required to test software. Testing ensures that software meets user specifications and requirements. However, the field of software testing has a number of underlying issues like effective generation of test cases, prioritisation of test cases etc which need to be tackled. These issues demand on effort, time and cost of the testing. Different techniques and methodologies have been proposed for taking care of these issues. Use of evolutionary algorithms…
Software vulnerabilities can have serious consequences, which is why many techniques have been proposed to defend against them. Among these, vulnerability detection techniques are a major area of focus. However, there is a lack of a comprehensive approach for benchmarking these proposed techniques. In this paper, we present the first survey that comprehensively investigates and summarizes the current state of software vulnerability detection benchmarking. We review the current literature on benchmarking vulnerability detection, including benchmarking approaches in technique-proposing papers an…
Software development, despite all the significant improvements it contributes to society, is a very expensive high-risk venture. Every software project commences with the intention to deliver a software product on time and within budget, but a great percentage of these software projects either fail to achieve their full potential or fail entirely. A lot of factors come to play in this regard and the dominant amongst them, which is often overlooked and oftentimes the biggest or strongest factor, is the human factor, as software development is a basic human endeavor. Knowledge and understanding …
Effort estimation is a key factor for software project success, defined as delivering software of agreed quality and functionality within schedule and budget. Traditionally, effort estimation has been used for planning and tracking project resources. Effort estimation methods founded on those goals typically focus on providing exact estimates and usually do not support objectives that have recently become important within the software industry, such as systematic and reliable analysis of causal effort dependencies. This article presents the results of a study of software effort estimation from…
Background: The need for empirical investigations in software engineering is growing. Many researchers nowadays, conduct and validate their solutions using empirical research. Survey is one empirical method which enables researchers to collect data from a large population. Main aim of the survey is to generalize the findings. Aims: In this study we aim to identify the problems researchers face during survey design, and mitigation strategies. Method: A literature review as well as semi-structured interviews with nine software engineering researchers were conducted to elicit their views on probl…
Determining the factors that have an influence on software systems development and deployment project outcomes has been the focus of extensive and ongoing research for more than 30 years. We provide here a survey of the research literature that has addressed this topic in the period 1996-2006, with a particular focus on empirical analyses. On the basis of this survey we present a new classification framework that represents an abstracted and synthesized view of the types of factors that have been asserted as influencing project outcomes.
Cloud computing is the next stage of the internet evolution. It relies on sharing of resources to achieve coherence on a network. It is emerged as new computing standard that impacts several different research fields, including software testing. There are various software techniques used for testing application. It not only changes the way of obtaining computing resources but also changes the way of managing and delivering computing services, technologies and solutions, meanwhile it causes new issues, challenges and needs in software testing. Software testing in cloud can reduce the need for h…
Context: Over the past decade Software Engineering research has seen a steady increase in survey-based studies, and there are several guidelines providing support for those willing to carry out surveys. The need for auditing survey research has been raised in the literature. Checklists have been used to assess different types of empirical studies, such as experiments and case studies. Objective: This paper proposes a checklist to support the design and assessment of survey-based research in software engineering grounded in existing guidelines for survey research. We further evaluated the check…
Context: Software Engineering research makes use of collections of software artifacts (corpora) to derive empirical evidence from. Goal: To improve quality and reproducibility of research, we need to understand the characteristics of used corpora. Method: For that, we perform a literature survey using grounded theory. We analyze the latest proceedings of seven relevant conferences. Results: While almost all papers use corpora of some kind with the common case of collections of source code of open-source Java projects, there are no frequently used projects or corpora across all the papers. For …
Past research on software product lines has focused on the initial development of reusable assets and related challenges, such as cost estimation and implementation issues. Naturally, as software product lines are increasingly adopted throughout industry, their ongoing maintenance and evolution are getting more attention as well. However, it is not clear to what degree research is following this trend, and where the interests and demands of the industry lie. In this technical report, we provide a survey and comparison of selected publications on software product line maintenance and evolution …
The software ecosystem is a trust-rich part of the world. Collaboratively, software engineers trust major hubs in the ecosystem, such as package managers, repository services, and programming language ecosystems. This trust, however, is often broken by vulnerabilities, ransomware, and abuse from malignant actors. But what is trust? In this paper we explore, through twelve in-depth interviews with software engineers, how they perceive trust in their daily work. From the interviews we conclude three things. First, software engineers make a distinction between an adoption factor and a trust facto…
Human values (e.g., pleasure, privacy, and social justice) are what a person or a society considers important. The inability to address them in software-intensive systems can result in numerous undesired consequences (e.g., financial losses) for individuals and communities. Various solutions (e.g., methodologies, techniques) are developed to help "operationalize values in software". The ultimate goal is to ensure building software (better) reflects and respects human values. In this survey, "operationalizing values" is referred to as the process of identifying human values and translating them…
Background: Research software plays an important role in solving real-life problems, empowering scientific innovations, and handling emergency situations. Therefore, the correctness and trustworthiness of research software are of absolute importance. Software testing is an important activity for identifying problematic code and helping to produce high-quality software. However, testing of research software is difficult due to the complexity of the underlying science, relatively unknown results from scientific algorithms, and the culture of the research software community. Aims: The goal of thi…
Service oriented architecture (SOA) is one of the latest software architectures. This architecture is created in direction of the business requirements and removed the gap between softwares and businesses. The software testing is the rising cost of activities in development software. SOA has different specifications and features proportion of the other software architectures. First this paper reviews SOA testing challenges and existing solution(s) for those challenges. Then that reports a survey of recent research to SOA systems testing, that covers both functional and non-functional testing. …
Background: Given the social aspects of Software Engineering (SE), in the last twenty years, researchers from the field started using research methods common in social sciences such as case study, ethnography, and grounded theory. More recently, case survey, another imported research method, has seen its increasing use in SE studies. It is based on existing case studies reported in the literature and intends to harness the generalizability of survey and the depth of case study. However, little is known on how case survey has been applied in SE research, let alone guidelines on how to employ it…
In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. The study was cond…
The Rational Unified Process a software engineering process is gaining popularity nowadays. RUP delivers best software practices for component software Development life cycle It supports component based software development. Risk is involved in every component development phase .neglecting those risks sometimes hampers the software growth and leads to negative outcome. In Order to provide appropriate security and protection levels, identifying various risks is very vital. Therefore Risk identification plays a very crucial role in the component based software development This report addresses i…
The Influence of Fraudulent AI-Generated Responses on Software Engineering Surveys
University of Calgary（卡尔加里大学）；University of Hawaii at Manoa（夏威夷大学马诺阿分校）；University of Bari（巴里大学）；Universidade Federal Rural de Pernambuco (UFRPE)（伯南布哥联邦农村大学）；Radboud University（拉德堡德大学）
Background: Studies on developer productivity and well-being find that the perceptions of productivity in a software team can be a socio-technical problem. Intuitively, problems and challenges can be better handled by managing expectations in software teams. Aim: Our goal is to understand whether the expectations of software developers vary towards diverse stakeholders in software teams. Method: We surveyed 181 professional software developers to understand their expectations from five different stakeholders: (1) organizations, (2) managers, (3) peers, (4) new hires, and (5) government and edu…
Today's significant technological advancement allows early-stage software startups to build and launch innovative products quickly on the market. However, many of them die in the early years of their path due to market conditions, ignorance of customer needs, lack of resources, or focus, such as the misuse of well-established practices. The study's motivation is to analyze software engineering practices in startups from a practitioner's perspective. Our objective was to identify practices and tools the startups employ in their daily routines. We carried out an expert survey study with 140 soft…
The Department of Homeland Security in the United States estimates that 90% of software vulnerabilities can be traced back to defects in design and software coding. The financial impact of these vulnerabilities has been shown to exceed 380 million USD in industrial control systems alone. Since software developers write software, they also introduce these vulnerabilities into the source code. However, secure coding guidelines exist to prevent software developers from writing vulnerable code. This study focuses on the human factor, the software developer, and secure coding, in particular secure …
Context: Software code review aims to early find code anomalies and to perform code improvements when they are less expensive. However, issues and challenges faced by developers who do not apply code review practices regularly are unclear. Goal: Investigate difficulties developers face to apply code review practices without limiting the target audience to developers who already use this practice regularly. Method: We conducted a web-based survey with 350 Brazilian practitioners engaged on the software development industry. Results: Code review practices are widespread among Brazilian practitio…
Logs are semi-structured text generated by logging statements in software source code. In recent decades, software logs have become imperative in the reliability assurance mechanism of many software systems because they are often the only data available that record software runtime information. As modern software is evolving into a large scale, the volume of logs has increased rapidly. To enable effective and efficient usage of modern software logs in reliability engineering, a number of studies have been conducted on automated log analysis. This survey presents a detailed overview of automate…
Runtime software patching aims to minimize or eliminate service downtime, user interruptions and potential data losses while deploying a patch. Due to modern software systems' high variance and heterogeneity, no universal solutions are available or proposed to deploy and execute patches at runtime. Existing runtime software patching solutions focus on specific cases, scenarios, programming languages and operating systems. This paper aims to identify, investigate and synthesize state-of-the-art runtime software patching approaches and gives an overview of currently unsolved challenges. It furth…
This year, the Agile Manifesto completes seventeen years and, throughout the world, companies and researchers seek to understand their adoption stage, as well as the benefits, barriers, and limitations of agile methods. Although we have some studies and questionnaire data at the global level, we know little about how the Paraguayan software community is adopting agile methods. The present work conducted a research to set up the current stage of adoption, initial concerns and barriers of implementation of agile methods in software development companies in Paraguay. An online survey was sent to …
Requirements Engineering (RE) is one of the prime areas in software development. Since agile software development englobes several emerging techniques and advocates for continuous improvement, it urges the question of which agile RE practices are currently most used, their characteristics, and the challenges in their employment. The aim of this work is to investigate and categorize the collection and specification of agile requirements practices based on how professionals perceive their importance for a software project that applies agile methodologies. Thus, a survey was carried out with fort…
Distribution can be a feature of the software evolution process. In other words, temporally and spatially distributed teams and organizations can develop and work on a software application. The simplest case is to outsource production and employ workforce at distributed sites so that multiple distributed teams can work on a project within a parallel framework. If this distribution is global, it will be called the global software evolution or development. A higher level of distribution is defined as decentralization and decentralized software evolution, which means that software development can…
