We present a longitudinal study on the long-term evolution of maintainability in open-source software. Quality assessment remains at the forefront of both software research and practice, with many models and assessment methodologies proposed and used over time. Some of them helped create and shape standards such as ISO 9126 and 25010, which are well established today. Both describe software quality in terms of characteristics such as reliability, security or maintainability. An important body of research exists linking these characteristics with software metrics, and proposing ways to automate quality assessment by aggregating software metric values into higher-level quality models. We employ the Maintainability Index, technical debt ratio and a maintainability model based on the ARiSA Compendium. Our study covers the entire 18 year development history and all released versions for three complex, open-source applications. We determine the maintainability for each version using the proposed models, we compare obtained results and use manual source code examination to put them into context. We examine the common development patterns of the target applications and study the relation between refactoring and maintainability. Finally, we study the strengths and weaknesses of each maintainability model using manual source code examination as the baseline.
We report a methodology developed to quantitatively assess the maintainability of Geant4 with respect to software engineering references. The level of maintainability is determined by combining a set of metrics values whose references are documented in literature.
Our study is focused on an evaluation of the maintainability characteristic in the context of the long-term evolution of open-source software. According to well established software quality models such as the ISO 9126 and the more recent ISO 25010, maintainability remains among key quality characteristics alongside performance, security and reliability. To achieve our objective, we selected three complex, widely used target applications for which access to their entire development history and source code was available. To enable cross-application comparison, we restricted our selection to GUI-…
The software is changing rapidly with the invention of advanced technologies and methodologies. The ability to rapidly and successfully upgrade software in response to changing business requirements is more vital than ever. For the long-term management of software products, measuring software maintainability is crucial. The use of soft computing techniques for software maintainability prediction has shown immense promise in software maintenance process by providing accurate prediction of software maintainability. To better understand the role of soft computing techniques for software maintaina…
Dependency injection (DI) is generally known to improve maintainability by keeping application classes separate from the library. Particularly within the Java environment, there are many applications using the principles of DI with the aim to improve maintainability. There exists some work that provides an inference on the impact of DI on maintainability, but no conclusive evidence is provided. The fact that there are no publicly available tools for quantifying DI makes such an evidence more difficult to be produced. In this paper, we propose a novel metric, DCBO, to measure the proportion of …
The technical state of software, i.e., its technical debt (TD) and maintainability are of increasing interest as ever more software is developed and deployed. Since td and maintainability are neither uniformly defined, not easy to understand, nor directly measurable, practitioners are likely to apply readily available tools to assess TD or maintainability and they may rely on the reported results without properly understanding what they embody. In this paper, we: a) methodically identify 11 readily available tools that measure TD or maintainability, b) present an in-depth investigation on how …
Maintainability is a key quality attribute of successful software systems. However, its management in practice is still problematic. Currently, there is no comprehensive basis for assessing and improving the maintainability of software systems. Quality models have been proposed to solve this problem. Nevertheless, existing approaches do not explicitly take into account the maintenance activities, that largely determine the software maintenance effort. This paper proposes a 2-dimensional model of maintainability that explicitly associates system properties with the activities carried out during…
Security is a requirement of utmost importance to produce high-quality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub's model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security an…
Automated Production Systems (aPS) have lifetimes of up to 30-50 years, throughout which the desired products change ever more frequently. This requires flexible, reusable control software that can be easily maintained and evolved. To evaluate selected criteria that are especially relevant for maturity in software maintainability and evolvability of aPS, the approach SWMAT4aPS+ builds on a questionnaire with 52 questions. The three main research questions cover updates of software modules and success factors for both cross-disciplinary development as well as reusable models. This paper present…
Dynamic Probabilistic Risk Assessment (DPRA) is a powerful concept that is used to evaluate design and safety of complex industrial systems. A DPRA model uses a conceptual system representation as a formal basis for simulation and analysis. In this paper we consider an adaptive maintenance of DPRA models that consist in modifying and extending a simplified model to a real-size DPRA model. We propose an approach for quantitative maintainability assessment of DPRA models created with an industrial modeling tool called PyCATSHOO. We review and adopt some metrics from conceptual modeling, software…
Advanced AI technologies are serving humankind in a number of ways, from healthcare to manufacturing. Advanced automated machines are quite expensive, but the end output is supposed to be of the highest possible quality. Depending on the agility of requirements, these automation technologies can change dramatically. The likelihood of making changes to automation software is extremely high, so it must be updated regularly. If maintainability is not taken into account, it will have an impact on the entire system and increase maintenance costs. Many companies use different programming paradigms i…
Large-scale simulations play a central role in science and the industry. Several challenges occur when building simulation software, because simulations require complex software developed in a dynamic construction process. That is why simulation software engineering (SSE) is emerging lately as a research focus. The dichotomous trade-off between scalability and efficiency (SE) on the one hand and maintainability and portability (MP) on the other hand is one of the core challenges. We report on the SE/MP trade-off in the context of an ongoing systematic literature review (SLR). After characteriz…
This paper reports the use of a qualitative methodology for conducting longitudinal case study research on software development. We provide a detailed description and explanation of appropriate methods of qualitative data collection and analysis that can be utilized by other researchers in the software engineering field. Our aim is to illustrate the utility of longitudinal case study research, as a complement to existing methodologies for studying software development, so as to enable the community to develop a fuller and richer understanding of this complex, multi-dimensional phenomenon. We d…
Internal software quality determines the maintainability of the software product and influences the quality in use. There is a plethora of metrics which purport to measure the internal quality of software, and these metrics are offered by static software analysis tools. To date, a number of reports have assessed the validity of these metrics. No data are available, however, on whether metrics offered by the tools are somehow validated in scientific studies. The current study covers this gap by providing data on which tools and how many validated metrics are provided. The results show that a ra…
Software metrics capture information about software development processes and products. These metrics support decision-making, e.g., in team management or dependency selection. However, existing metrics tools measure only a snapshot of a software project. Little attention has been given to enabling engineers to reason about metric trends over time -- longitudinal metrics that give insight about process, not just product. In this work, we present PRiME (PRocess MEtrics), a tool for computing and visualizing process metrics. The currently-supported metrics include productivity, issue density, is…
This paper presents a tertiary review of software quality measurement research. To conduct this review, we examined an initial dataset of 7,811 articles and found 75 relevant and high-quality secondary analyses of software quality research. Synthesizing this body of work, we offer an overview of perspectives, measurement approaches, and trends. We identify five distinct perspectives that conceptualize quality as heuristic, as maintainability, as a holistic concept, as structural features of software, and as dependability. We also identify three key challenges. First, we find widespread evidenc…
Early prediction of software quality is important for better software planning and controlling. In early development phases, design complexity metrics are considered as useful indicators of software testing effort and some quality attributes. Although many studies investigate the relationship between design complexity and cost and quality, it is unclear what we have learned beyond the scope of individual studies. This paper presented a systematic review on the influence of software complexity metrics on quality attributes. We aggregated Spearman correlation coefficients from 59 different data …
In the software development industry, technical debt is regarded as a critical issue in term of the negative consequences such as increased software development cost, low product quality, decreased maintainability, and slowed progress to the long-term success of developing software. However, despite the vast research contributions in technical debt management for software engineering, the idea of technical debt fails to provide a holistic consideration to include both IT and business aspects. Further, implementing an enterprise architecture (EA) project might not always be a success due to unc…
Background: Test-Driven Development (TDD) is an agile software development practice, which is claimed to boost both external quality of software products and developers' productivity. Aims: We want to study (i) the TDD effects on the external quality of software products as well as the developers' productivity, and (ii) the retainment of TDD over a period of five months. Method: We conducted a (quantitative) longitudinal cohort study with 30 third year undergraduate students in Computer Science at the University of Bari in Italy. Results: The use of TDD has a statistically significant effect n…
Modifications to open-source software (OSS) are often provided in the form of "patch stacks" - sets of changes (patches) that modify a given body of source code. Maintaining patch stacks over extended periods of time is problematic when the underlying base project changes frequently. This necessitates a continuous and engineering-intensive adaptation of the stack. Nonetheless, long-term maintenance is an important problem for changes that are not integrated into projects, for instance when they are controversial or only of value to a limited group of users. We present and implement a methodolo…
Software ecosystems (SECOs) and open innovation processes have been claimed as a way forward for the software industry. A proper understanding of requirements is as important for these IT-systems as for more traditional ones. This paper presents a mapping study on the issues of requirements engineering and quality aspects in SECOs and analyzes emerging ideas. Our findings indicate that among the various phases or subtasks of requirements engineering, most of the SECO specific research has been accomplished on elicitation, analysis, and modeling. On the other hand, requirements selection, prior…
Technical Debt is a term begat by Ward Cunningham to signify the measure of adjust required to put a software into that state which it ought to have had from the earliest starting point. Often organizations need to support continuous and fast delivery of customer value both in short and a long-term perspective and later have to compromise with the quality and productivity of the software. So, a simple solution could be to repay the debts as and when they are encountered to avoid maintainability cost and subsequent delays. Therefore, it has become inevitable to identify and come up with techniq…
Code smells are symptoms of poor design and implementation choices, which might hinder comprehension, increase code complexity and fault-proneness and decrease maintainability of software systems. The aim of our study was to perform a triangulation of bibliometric and thematic analysis on code smell literature production. The search was performed on Scopus (Elsevier, Netherlands) database using the search string code smells which resulted in 442 publications. The Go-to statement was the first bad code smells identified in software engineering history in 1968. The literature production trend ha…
Architectural smells (AS) are notorious for their long-term impact on the Maintainability and Evolvability of software systems. The majority of research work has investigated this topic by mining software repositories of open source Java systems, making it hard to generalise and apply them to an industrial context and other programming languages. To address this research gap, we conducted an embedded multiple-case case study, in collaboration with a large industry partner, to study how AS evolve in industrial embedded systems. We detect and track AS in 9 C/C++ projects with over 30 releases fo…
