Toward a Benchmark Repository for Software Maintenance Tool Evaluations with Humans
Matúš Sulír
发表信息：
Proceedings of the 2019 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH Companion '19), ACM, 2019, pp. 7-8
To evaluate software maintenance techniques and tools in controlled experiments with human participants, researchers currently use projects and tasks selected on an ad-hoc basis. This can unrealistically favor their tool, and it makes the comparison of results difficult. We suggest a gradual creation of a benchmark repository with projects, tasks, and metadata relevant for human-based studies. In this paper, we discuss the requirements and challenges of such a repository, along with the steps which could lead to its construction.
Software repository mining is the foundation for many empirical software engineering studies. The collection and analysis of detailed data can be challenging, especially if data shall be shared to enable replicable research and open science practices. SmartSHARK is an ecosystem that supports replicable and reproducible research based on software repository mining.
This paper proposes a software repository model together with associated tooling and consists of several complex, open-source GUI driven applications ready to be used in empirical software research. We start by providing the rationale for our repository and criteria that guided us in searching for suitable applications. We detail the model of the repository together with associated artifacts and supportive tooling. We detail current applications in the repository together with ways in which it can be further extended. Finally we provide examples of how our repository facilitates research in so…
The way practitioners perform maintenance tasks in practice is little known by researchers. In turn, practitioners are not always up to date with the proposals provided by the research community. This work investigates the gap between software maintenance techniques proposed by the research community and the software maintenance practice. We carried out a survey with 112 practitioners from 92 companies and 12 countries. We concentrate on analyzing if and how practitioners understand and apply the following subjects: bad smells, refactoring, software metrics, and change impact analysis. This st…
Lehman's Laws teach us that a software system will become progressively less satisfying to its users over time, unless it is continually adapted to meet new needs. A line of previous works sought to better understand software maintenance by studying how commits can be classified into three main software maintenance activities. Corrective: fault fixing; Perfective: system improvements; Adaptive: new feature introduction. In this work we suggest visualizations for exploring software maintenance activities in both project and individual developer scopes. We demonstrate our approach using a protot…
Maintenance is the last and the most critical phase of the software development life cycle. It involves debugging of errors and different types of enhancements which are requested by the user. Software reliability regarding maintenance is the most crucial part as it depends upon the time and cost to correct the errors and make enchantements. It is often felt that software errors or correction takes time to be removed. The maintenance time depends upon the nature of the occurred errors and requested enhancements. In this research work we predict the software reliability in terms of time taken t…
Easy and mostly free access to the internet has resulted in the growing use of open source software (OSS). However, it is a common perception that closed proprietary software is still superior in areas such as software maintenance and management. The research model of this study establishes a relationship between maintenance issues (such as user requests and error handling) and support provided by open source software through project forums, mailing lists and trackers. To conduct this research, we have used a dataset consisting of 120 open source software projects, covering a wide range of cat…
Software traceability plays a critical role in software maintenance and evolution. We conducted a systematic mapping study with six research questions to understand the benefits, costs, and challenges of using traceability in maintenance and evolution. We systematically selected, analyzed, and synthesized 63 studies published between January 2000 and May 2020, and the results show that: traceability supports 11 maintenance and evolution activities, among which change management is the most frequently supported activity; strong empirical evidence from industry is needed to validate the impact o…
From the preliminary stage of software engineering, selection of appropriate enforcement of standards remained a challenge for stakeholders during entire cycle of software development, but it can lead to reduce the efforts desired for software maintenance phase. Corrective maintenance is the reactive modification of a software product performed after delivery to correct discovered faults. Studies conducted by different researchers reveal that approximately 50 to 75 percent of the effort is spent on maintenance, out of which about 17 to 21 percent is exercised on corrective maintenance. In this…
Many software engineering studies or tasks rely on categorizing software engineering artifacts. In practice, this is done either by defining simple but often imprecise heuristics, or by manual labelling of the artifacts. Unfortunately, errors in these categorizations impact the tasks that rely on them. To improve the precision of these categorizations, we propose to gather heuristics in a collaborative heuristic repository, to which researchers can contribute a large amount of diverse heuristics for a variety of tasks on a variety of SE artifacts. These heuristics are then leveraged by state-o…
Reusing previously completed software repository to enhance the development process is a common phenomenon. If developers get suggestions from the existing projects they might be benefited a lot what they eventually expect while coding. The strategies available in this field have been rapidly changing day by day. There are a number of efforts that have been focusing on mining process and constructing repository. Some of them have emphasized on the web based code searching while others have integrated web based code searching in their customized tool. But web based approaches have inefficiency …
Lehman's Laws teach us that a software system will become progressively less satisfying to its users over time, unless it is continually adapted to meet new needs. Understanding software maintenance can potentially relieve many of the pains currently experienced by practitioners in the industry and assist in reducing uncertainty, improving cost-effectiveness, reliability and more. The research community classifies software maintenance into 3 main activities: Corrective: fault fixing; Perfective: system improvements; Adaptive: new feature introduction. In this work we seek to model software mai…
Fixing bugs is an important phase in software development and maintenance. In practice, the process of bug fixing may conflict with the release schedule. Such confliction leads to a trade-off between software quality and release schedule, which is known as the technical debt metaphor. In this article, we propose the concept of debt-prone bugs to model the technical debt in software maintenance. We identify three types of debt-prone bugs, namely tag bugs, reopened bugs, and duplicate bugs. A case study on Mozilla is conducted to examine the impact of debt-prone bugs in software products. We inv…
Software systems are getting more complex as the system grows where maintaining such system is a primary concern for the industry. Code clone is one of the factors making software maintenance more difficult. It is a process of replicating code blocks by copy-and-paste that is common in software development. In the beginning stage of the project, developers find it easy and time consuming though it has crucial drawbacks in the long run. There are two types of researchers where some researchers think clones lead to additional changes during maintenance phase, in later stage increase the overall …
This paper presents an analysis of technical debt management through resources allocation policies in software maintenance process during its operation to demonstrate how different strategies leads to the emergence of different behaviors along the evolution path. To achieve this objective, this work used the System Dynamic approach for building a computational simulation model based on extensive literature review and secondary data. Most of the works that applied the System Dynamics on software projects research, focused on initial phases of its life cycle, leaving a gap to be explored regardi…
Automatic testing is a widely adopted technique for improving software quality. Software developers add, remove and update test methods and test classes as part of the software development process as well as during the evolution phase, following the initial release. In this work we conduct a large scale study of 61 popular open source projects and report the relationships we have established between test maintenance, production code maintenance, and semantic changes (e.g, statement added, method removed, etc.). performed in developers' commits. We build predictive models, and show that the num…
Background: Data mining and analyzing of public Git software repositories is a growing research field. The tools used for studies that investigate a single project or a group of projects have been refined, but it is not clear whether the results obtained on such ``convenience samples'' generalize. Aims: This paper aims to elucidate the difficulties faced by researchers who would like to ascertain the generalizability of their findings by introducing an interface that addresses the issues with obtaining representative samples. Results: To do that we explore how to exploit the World of Code syst…
Popular (re)use of third-party open-source software (OSS) is evidence of the impact of hosting repositories like maven on software development today. Updating libraries is crucial, with recent studies highlighting the associated vulnerabilities with aging OSS libraries. The decision to migrate to a newer library can range from trivial (security threat) to complex (assessment of work required to accommodate the changes). By leveraging the `wisdom of the software repository crowd' we propose a simple and efficient approach to recommending `consented' library updates. Our Software Universe Graph …
Developers use and contribute to repositories on GitHub. Documentation present in the repositories serves as an important source by helping developers to understand, maintain and contribute to the project. Currently, documentation in a repository is diversified, among various files, with most of it present in ReadMe files. However, other software artifacts in the repository, such as issue reports and pull requests could also contribute to documentation, without documentation being explicitly specified. Hence, in this paper, we propose a taxonomy of documentation sources by analyzing different …
Mining and storage of data from software repositories is typically done on a per-project basis, where each project uses a unique combination of data schema, extraction tools, and (intermediate) storage infrastructure. We introduce GraphRepo, a tool that enables a unified approach to extract data from Git repositories, store it, and share it across repository mining projects. GraphRepo usesNeo4j, an ACID-compliant graph database management system, and allows modular plug-in of components for repository extraction (drillers), analysis (miners), and export (mappers). The graph enables a natural w…
Software development is getting changed so rapidly. It will be highly benefited if we can accelerate software development process by guiding developers. Appropriate guidelines and accurate recommendations to developers during development process can reduce software development expenses, as well as can save valuable times of developers. There are a number of approaches to speed up the software development process. It can be done through code assistance tools that help developers by recommending relevant items from searching particular repository of Application Programming Interface (API). Some …
We advocate for a paradigm shift in supporting free/libre and open source software (FLOSS) ecosystem maintenance, from focusing on individual projects to monitoring a whole organic system of the entire FLOSS ecosystem, which we call software meta-maintenance. We discuss challenges of building a global source code management system, a global issue management system, and FLOSS human capital index, based on the blockchain technologies.
This paper describes a set of tools for automating and controlling the development and maintenance of software systems. The mental model is a software assembly line. Program design and construction take place at individual programmer workstations. Integration of individual software components takes place at subsequent stations on the assembly line. Software is moved automatically along the assembly line toward final packaging. Software under construction or maintenance is divided into packages. Each package of software is composed of a recipe and ingredients. Some new terms are introduced to d…
The technical debt (TD) metaphor is widely used to encapsulate numerous software quality problems. She describes the trade-off between the short term benefit of taking a shortcut during the design or implementation phase of a software product (for example, in order to meet a deadline) and the long term consequences of taking said shortcut, which may affect the quality of the software product. TDs must be managed to guarantee the software quality and also reduce its maintenance and evolution costs. However, the tools for TD detection usually provide results only considering the files perspectiv…
Universiti Malaysia Terengganu（马来西亚登嘉楼大学）；The University of Lahore（拉合尔大学）；Bahria University（巴里亚大学）
In software engineering, software maintenance is the process of correction, updating, and improvement of software products after handed over to the customer. Through offshore software maintenance outsourcing clients can get advantages like reduce cost, save time, and improve quality. In most cases, the OSMO vendor generates considerable revenue. However, the selection of an appropriate proposal among multiple clients is one of the critical problems for OSMO vendors. The purpose of this paper is to suggest an effective machine learning technique that can be used by OSMO vendors to assess or pre…
Background. The software architecture recovery method RELAX produces a concern-based architectural view of a software system graphically and textually from that system's source code. The method has been implemented in software which can be run on subject systems whose source code is written in Java. Aims. Our aim was to find out whether the availability of architectural views produced by RELAX can help maintainers who are new to a project in becoming productive with development tasks sooner, and find out how they felt about working in such an environment. Method. We conducted a user study with…
The recent progress of artificial intelligence(AI) has shown great potentials for alleviating human burden in various complex tasks. From the view of software engineering, AI techniques can be seen in many fundamental aspects of development, such as source code comprehension, in which state-of-the-art models are implemented to extract and express the meaning of code snippets automatically. However, such technologies are still struggling to tackle and comprehend the complex structures within industrial code, thus far from real-world applications. In the present work, we built an innovative and …
Outdated documentation is a pervasive problem in software development, preventing effective use of software, and misleading users and developers alike. We posit that one possible reason why documentation becomes out of sync so easily is that developers are unaware of when their source code modifications render the documentation obsolete. Ensuring that the documentation is always in sync with the source code takes considerable effort, especially for large codebases. To address this situation, we propose an approach that can automatically detect code element references that survive in the docume…
Situational method engineering uses a repository of reusable method fragments that are derived from existing software development methodologies and industrial best practices to simplify the construction of any project-specific software development methodology aligned with specific characteristics of a project at hand. In this respect, OPEN is a well-established, standardized and popular approach for situational method engineering. It has a large repository of reusable method fragments called OPF that method engineers can select and assemble them according to the requirements of a project to co…
In order to maintain a system, it is beneficial to know its software architecture. In the common case that this architecture is unavailable, architecture recovery provides a way to recover an architectural view of the system. Many different methods and tools exist to provide such a view. While there have been taxonomies of different recovery methods and surveys of their results along with measurements of how these results conform to expert's opinions on the systems, there has not been a survey that goes beyond an automatic comparison and instead seeks to answer questions about the viability of…
The bug growth pattern prediction is a complicated, unrelieved task, which needs considerable attention. Advance knowledge of the likely number of bugs discovered in the software system helps software developers in designating sufficient resources at a convenient time. The developers may also use such information to take necessary actions to increase the quality of the system and in turn customer satisfaction. In this study, we examine eight different time series forecasting models, including Long Short Term Memory Neural Networks (LSTM), auto-regressive integrated moving average (ARIMA), and …
Service-orientation is a promising paradigm that enables the engineering of large-scale distributed software systems using rigorous software development processes. The existing problem is that every service-oriented software development project often requires a customized development process that provides specific service-oriented software engineering tasks in support of requirements unique to that project. To resolve this problem and allow situational method engineering, we have defined a set of method fragments in support of the engineering of the project-specific service-oriented software d…
Many empirical software engineering studies show that there is a great need for repositories where source code is acquired, filtered and classified. During the last few years, Ethereum block explorer services have emerged as a popular project to explore and search Ethereum blockchain data such as transactions, addresses, tokens, smart-contracts' source code, prices and other activities taking place on the Ethereum blockchain. Despite the availability of this kind of services, retrieving specific information useful to empirical software engineering studies, such as the study of smart-contracts'…
