The widespread adoption of Free/Libre and Open Source Software (FLOSS) means that the ongoing maintenance of many widely used software components relies on the collaborative effort of volunteers who set their own priorities and choose their own tasks. We argue that this has created a new form of risk that we call 'underproduction' which occurs when the supply of software engineering labor becomes out of alignment with the demand of people who rely on the software produced. We present a conceptual framework for identifying relative underproduction in software as well as a statistical method for applying our framework to a comprehensive dataset from the Debian GNU/Linux distribution that includes 21,902 source packages and the full history of 461,656 bugs. We draw on this application to present two experiments: (1) a demonstration of how our technique can be used to identify at-risk software packages in a large FLOSS repository and (2) a validation of these results using an alternate indicator of package risk. Our analysis demonstrates both the utility of our approach and reveals the existence of widespread underproduction in a range of widely-installed software components in Debian.
Cooperative learning theory has shown that stable membership is a hallmark of effective work teams. According to relation strength and social network centrality, this paper proposes an approach to measure team stability reasons in consecutive cooperative learning. Taking consecutive course projects of software engineering in a university as examples, we examine the relation between team stability and learning performance in consecutive cooperative learning from two parts: learning score and learning satisfaction. Through empirical analysis, it arrives at the conclusion that learning score is i…
The amount of energy consumed during the execution of software, and the ability to predict future consumption, is an important factor in the design of embedded electronic systems. In this technical report I examine factors in the execution of software that can affect energy consumption. Taking a simple embedded software benchmark I measure to what extent input data can affect energy consumption, and propose a method for reflecting this in software energy models.
The summary presented in this paper highlights the results obtained in a four-years project aiming at analyzing the development process of software artifacts from two points of view: Effectiveness and Affectiveness. The first attribute is meant to analyze the productivity of the Open Source Communities by measuring the time required to resolve an issue, while the latter provides a novel approach for studying the development process by analyzing the affectiveness ex-pressed by developers in their comments posted during the issue resolution phase. Affectivenes is obtained by measuring Sentiment,…
In this work, we briefly introduce and discuss some of the diversity measures used in Ecology. After a succinct description and analysis of the most relevant ones, we single out the Shannon-Weiner index. We justify why it is the most informative and relevant one for measuring software diversity. Then, we show how it can be used for effectively assessing the diversity of various real software ecosystems. We discover in the process a frequently overlooked software monopoly, and its key security implications. We finally extract some conclusions from the results obtained, focusing mostly on their …
Software implements a significant proportion of functionality in factory automation. Thus, efficient development and the reuse of software parts, so-called units, enhance competitiveness. Thereby, complex control software units are more difficult to understand, leading to increased development, testing and maintenance costs. However, measuring complexity is challenging due to many different, subjective views on the topic. This paper compares different complexity definitions from literature and considers with a qualitative questionnaire study the complexity perception of domain experts, who con…
Software quality in use comprises quality from the user's perspective. It has gained its importance in e-government applications, mobile-based applications, embedded systems, and even business process development. User's decisions on software acquisitions are often ad hoc or based on preference due to difficulty in quantitatively measuring software quality in use. But, why is quality-in-use measurement difficult? Although there are many software quality models, to the authors' knowledge no works survey the challenges related to software quality-in-use measurement. This article has two main con…
The B method has facilitated the development of software by specifying the design of software as abstract machines and formally verifying the correctness of the abstract machines. The quality of B abstract machines can significantly impact the quality of final software products. In this paper, we propose a set of criteria for measuring the quality of B abstract machines based on ISO/IEC 25010, which is one of the latest international standards for evaluating software quality in software engineering. These criteria evaluate abstract machines using a number of general-purpose and domain-independ…
It is well known fact that was phrased by famous quality scholar P.B. Crosby that it is always cheaper to do the job right the first time. However, this statement must be reconsidered with respect to software development projects, because the concept of quality and associated costs measurements in software engineering discipline is not as matured as in manufacturing and other fields of the industry. Post delivery defects (i.e. software bugs) are very common and integral part of software industry. While the process of measuring and classifying quality cost components is visible, obvious and ins…
Software engineering is a human activity. Despite this, human aspects are under-represented in technical debt research, perhaps because they are challenging to evaluate. This study's objective was to investigate the relationship between technical debt and affective states (feelings, emotions, and moods) from software practitioners. Forty participants (N = 40) from twelve companies took part in a mixed-methods approach, consisting of a repeated-measures (r = 5) experiment (n = 200), a survey, and semi-structured interviews. The statistical analysis shows that different design smells (strong ind…
In the software development required a fidelity and accuracy in determining the size or value of the software to fit the operation is executed. Various methods of calculation has been widely applied to estimate the size, and one of them is by using the method of Function Point Analysis (FPA). The method is then applied by author to measure the complexity of an academic information system by using the two modeling approaches, namely object oriented and structural models. Measurements in this paper consists of several stages, namely describing the information system that will be built into the U…
Variability-aware metrics are designed to measure qualitative aspects of software product lines. As we identified in a prior SLR \cite{El-SharkawyYamagishi-EichlerSchmid19}, there exist already many metrics that address code or variability separately, while the combination of both has been less researched. MetricHaven fills this gap, as it extensively supports combining information from code files and variability models. Further, we also enable the combination of well established single system metrics with novel variability-aware metrics, going beyond existing variability-aware metrics. Our to…
It has recently become increasingly important to measure the cognitive load of developers. This is because continuing with a development under a high cognitive load may cause human errors. Therefore, to automatically measure the cognitive load, existing studies have used biometric measures such as brain waves and the heart rate. However, developers are often required to equip certain devices when measuring them, and can therefore be physically burdened. In this study, we evaluated the feasibility of non-invasive biometric measures based on the nasal skin temperature. The nasal skin temperature…
Quality managers, change agents and researchers are often troubled in defining and demonstrating the level of success achieved in software process improvement (SPI) initiatives. So far, there exist only few frameworks for identifying the level of success achieved in SPI. Analysis shows that these frameworks do not provide a comprehensive view from all relevant stakeholders involved in SPI. Early results from an ongoing research effort to discover and operationalise success dimensions are reported. Adapted from the project management literature it is suggested that five dimensions characterise …
As the modern vehicle becomes more software-defined, it is beginning to take significant effort to avoid serious regression in software design. This is because automotive software architects rely largely upon manual review of code to spot deviations from specified design principles. Such an approach is both inefficient and prone to error. In recent days, neural language models pre-trained on source code are beginning to be used for automating a variety of programming tasks. In this work, we extend the application of such a Programming Language Model (PLM) to automate the assessment of design c…
The Software Infrastructure Attitude Scale (SIAS): A Questionnaire Instrument for Measuring Professionals' Attitudes Toward Technical and Sociotechnical Infrastructure
A product line approach can save valuable resources by reusing artifacts. Especially for software artifacts, the reuse of existing components is highly desirable. In recent literature, the creation of software product lines is mainly proposed from a top-down point of view regarding features which are visible by customers. In practice, however, the design for a product line often arises from one or few existing products that descend from a very first product starting with copy-paste and evolving individually. In this contribution, we propose the theoretical basis to derive a set of metrics for …
The idea of automatizing the assessment of objectoriented design is not new. Different approaches define and apply their own quality models, which are composed of single metrics or combinations thereof, to operationalize software design. However, single metrics are too fine-grained to identify core design flaws and they cannot provide hints for making design improvements. In order to deal with these weaknesses of metric-based models, rules-based approaches have proven successful in the realm of source-code quality. Moreover, for developing a well-designed software system, design principles pla…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
This paper defines software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates, focusing on causality in discriminatory behavior. Evidence of software discrimination has been found in modern software systems that recommend criminal sentences, grant access to financial products, and determine who is allowed to participate in promotions. Our approach, Themis, generates efficient test suites to measure discrimination. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not req…
We apply machine learning to version control data to measure the quantity of effort required to produce source code changes. We construct a model of a `standard coder' trained from examples of code changes produced by actual software developers together with the labor time they supplied. The effort of a code change is then defined as the labor hours supplied by the standard coder to produce that change. We therefore reduce heterogeneous, structured code changes to a scalar measure of effort derived from large quantities of empirical data on the coding behavior of software developers. The stand…
Context: Over the last decades, open-source software has pervaded the software industry and has become one of the key pillars in software engineering. The incomparable growth of open source reflected that pervasion: Prior work described open source as a whole to be growing linearly, polynomially, or even exponentially. Objective: In this study, we explore the long-term growth of open source and corroborating previous findings by replicating previous studies on measuring the growth of open source projects. Method: We replicate four existing measurements on the growth of open source on a sample …
Measurement-based analysis of software timing behavior provides important insight and evidence for flight certification of modern avionics systems. For multicore systems, however, this analysis is challenging due to interference effects from shared hardware resource usage. We present an approach to multicore timing analysis that uses interference generators to stress interference channels in multicore systems. The test methodology comprises two steps. First, platform characterization measures the sensitivity of the hardware and RTOS to targeted interference channels using a combination of inte…
With even the most trivial of applications now being written on top of millions of lines code of libraries, API's, and programming languages, much of the complexity that used to exist when designing software has been abstracted away to allow programmers to focus on primarily business logic. With each application relying so heavily on the ecosystem it was designed to run in, whether that is limited to a local system or includes dependencies on machines connected by networks, measuring the complexity of these systems can no longer be done simply by observing the code internal to the application;…
It is well-known, and often a topic of heated debates, that programs in some programming languages are more concise than in others. This is a relevant factor when comparing or aggregating volume-impacted metrics on source code written in a combination of programming languages. In this paper, we present a model for measuring the conciseness of programming languages in a consistent, objective and evidence-based way. We present the approach, explain how it is founded on information theoretical principles, present detailed analysis steps and show the quantitative results of applying this model to …
Many tools have been created for measuring the agility of software teams, thus creating a saturation in the field. Three agile measurement tools were selected in order to validate whether they yield sim-ilar results. The surveys of the tools were given to teams in Company A (N = 30). The questions were grouped into agile practices which were checked for correlation in order to establish convergent validity. In addition, we checked whether the questions identified to be the same among the tools would be given the same replies by the respondents. We could not establish convergent validity since …
Many software developers rely on open source software for developing their applications and writing their source codes. Measuring an independent project's overall productivity is still an open problem for many technology companies. In this project, we address to bridge the gap of analyzing which are the most important features for prediction of a productivity based system. We have chosen to collect data from GitHub via their application programming interfaces (API) and analyze the data we gathered to understand the relation between the average time to close an issue and the features that we co…
As modern software extensively uses free open source packages as dependencies, developers have to regularly pull in new third-party code through frequent updates. However, without a proper review of every incoming change, vulnerable and malicious code can sneak into the codebase through these dependencies. The goal of this study is to aid developers in securely accepting dependency updates by measuring if the code changes in an update have passed through a code review process. We implement Depdive, an update audit tool for packages in Crates.io, npm, PyPI, and RubyGems registry. Depdive first …
