In computational science and in computer science, research software is a central asset for research. Computational science is the application of computer science and software engineering principles to solving scientific problems, whereas computer science is the study of computer hardware and software design.
The Open Science agenda holds that science advances faster when we can build on existing results. Therefore, research software has to be reusable for advancing science. Thus, we need proper research software engineering for obtaining reusable and sustainable research software. This way, software engineering methods may improve research in other disciplines. However, research in software engineering and computer science itself will also benefit from reuse when research software is involved.
For good scientific practice, the resulting research software should be open and adhere to the FAIR principles (findable, accessible, interoperable and repeatable) to allow repeatability, reproducibility, and reuse. Compared to research data, research software should be both archived for reproducibility and actively maintained for reusability. The FAIR data principles do not require openness, but research software should be open source software. Established open source software licenses provide sufficient licensing options, such that it should be the rare exception to keep research software closed.
We review and analyze the current state in this area in order to give recommendations for making computer science research software FAIR and open. We observe that research software publishing practices in computer science and in computational science show significant differences.
Machine learning software is increasingly being used to make decisions that affect people's lives. But sometimes, the core part of this software (the learned model), behaves in a biased manner that gives undue advantages to a specific group of people (where those groups are determined by sex, race, etc.). This "algorithmic discrimination" in the AI software systems has become a matter of serious concern in the machine learning and software engineering community. There have been works done to find "algorithmic bias" or "ethical bias" in the software system. Once the bias is detected in the AI s…
University of Calgary（卡尔加里大学）；University College London（伦敦大学学院）；University of Otago（奥塔哥大学）；Federal Rural University of Pernambuco（伯南布哥联邦农村大学）；Radboud University（拉德堡德大学）
This document captures the discussion and deliberation of the FAIR for Research Software (FAIR4RS) subgroup that took a fresh look at the applicability of the FAIR Guiding Principles for scientific data management and stewardship for research software. We discuss the vision of research software as ideally reproducible, open, usable, recognized, sustained and robust, and then review both the characteristic and practiced differences of research software and data. This vision and understanding of initial conditions serves as a backdrop for an attempt at translating and interpreting the guiding pr…
Machine learning software is being used in many applications (finance, hiring, admissions, criminal justice) having a huge social impact. But sometimes the behavior of this software is biased and it shows discrimination based on some sensitive attributes such as sex, race, etc. Prior works concentrated on finding and mitigating bias in ML models. A recent trend is using instance-based model-agnostic explanation methods such as LIME to find out bias in the model prediction. Our work concentrates on finding shortcomings of current bias measures and explanation methods. We show how our proposed m…
We assert that it is the ethical duty of software engineers to strive to reduce software discrimination. This paper discusses how that might be done. This is an important topic since machine learning software is increasingly being used to make decisions that affect people's lives. Potentially, the application of that software will result in fairer decisions because (unlike humans) machine learning software is not biased. However, recent results show that the software within many data mining packages exhibits "group discrimination"; i.e. their decisions are inappropriately affected by "protecte…
Software plays a central role in scientific discovery. Improving how we develop and use software for research can have both broad and deep impacts on a spectrum of challenges and opportunities society faces today. The emergence of Research Software Engineer (RSE) as a role correlates with the growing complexity of scientific challenges and diversity of software team skills. In this paper, we describe research software science (RSS), an idea related to RSE, and particularly suited to research software teams. RSS promotes the use of scientific methodologies to explore and establish broadly appli…
As software engineering research becomes more concerned with the psychological, sociological and managerial aspects of software development, relevant theories from reference disciplines are increasingly important for understanding the field's core phenomena of interest. However, the degree to which software engineering research draws on relevant social sciences remains unclear. This study therefore investigates the use of social science theories in five influential software engineering journals over 13 years. It analyzes not only the extent of theory use but also what, how and where these theo…
Background: Assessing and communicating software engineering research can be challenging. Design science is recognized as an appropriate research paradigm for applied research but is seldom referred to in software engineering. Applying the design science lens to software engineering research may improve the assessment and communication of research contributions. Aim: The aim of this study is 1) to understand whether the design science lens helps summarize and assess software engineering research contributions, and 2) to characterize different types of design science contributions in the softwa…
The development of scientific software is often a partnership between domain scientists and scientific software engineers. It is especially important to embrace these collaborations when developing advanced scientific software, where sustainability, reproducibility, and extensibility are important. In the ideal case, as discussed in this manuscript, this brings together teams composed of the world's foremost scientific experts in a given field with seasoned software developers experienced in forming highly collaborative teams working on software to further scientific research.
Many fields of science rely on software systems to answer different research questions. For valid results researchers need to trust the results scientific software produces, and consequently quality assurance is of utmost importance. In this paper we are investigating the impact of quality assurance in the domain of computational materials science (CMS). Based on our experience in this domain we formulate challenges for validation and verification of scientific software and their results. Furthermore, we describe directions for future research that can potentially help dealing with these chall…
This article summarizes motivations, organization, and activities of the Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE5.1) held in Manchester, UK in September 2017. The WSSSPE series promotes sustainable research software by positively impacting principles and best practices, careers, learning, and credit. This article discusses the Code of Conduct, idea papers, position papers, experience papers, demos, and lightning talks presented during the workshop. The main part of the article discusses the speed-blogging groups that formed during the meeting, along with …
Given the data-intensive and collaborative trend in science, the software engineering community also pays increasing attention to obtaining valuable and useful insights from data repositories. Nevertheless, applying data science to software engineering (e.g., mining software repositories) can be blindfold and meaningless, if lacking a suitable knowledge architecture (KA). By observing that software engineering practices are generally recorded through a set of factors (e.g., programmer capacity, different environmental conditions, etc.) involved in various software project aspects, we propose a…
This white paper argues that formal methods need to be better rooted in higher education curricula for computer science and software engineering programmes of study. To this end, it advocates (i) improved teaching of formal methods; (ii) systematic highlighting of formal methods within existing, `classical' computer science courses; and (iii) the inclusion of a compulsory formal methods course in computer science and software engineering curricula. These recommendations are based on the observations that (a) formal methods are an essential and cost-effective means to increase software quality;…
Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we sh…
The Science Gateways Community Institute (SGCI) is an NSF Software Infrastructure for Sustained Innovation (S2I2) funded project that leads and supports the science gateway community. Major activities for SGCI include a) sustainability training, including the Focus Week week-long course designed to help science gateway operators develop sustainability plans, and the Jumpstart virtual short-course; b) usability and user experience consulting; c) a community catalog of science gateways and science gateway software; d) workforce development activities, including a coding institute for students, i…
This technical report discusses the submission and peer-review process used by the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2) and the results of that process. It is intended to record both the alternative submission and program organization model used by WSSSPE2 as well as the papers associated with the workshop that resulted from that process.
How should software engineering be adapted for Computational Science (CS)? If we understood that, then we could better support software sustainability, verifiability, reproducibility, comprehension, and usability for CS community. For example, improving the maintainability of the CS code could lead to: (a) faster adaptation of scientific project simulations to new and efficient hardware (multi-core and heterogeneous systems); (b) better support for larger teams to co-ordinate (through integration with interdisciplinary teams); and (c) an extended capability to model complex phenomena. In order…
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific publications, shifts the focus away from sustainable research software development and reproducible results. The neglect of RSE in CSE at University research groups negatively impacts the scientific output: research data - including research software - related to a CSE publication cannot be found, reproduced, or re-used, different ideas are not combined easily into…
Background: Meeting the growing industry demand for Data Science requires cross-disciplinary teams that can translate machine learning research into production-ready code. Software engineering teams value adherence to coding standards as an indication of code readability, maintainability, and developer expertise. However, there are no large-scale empirical studies of coding standards focused specifically on Data Science projects. Aims: This study investigates the extent to which Data Science projects follow code standards. In particular, which standards are followed, which are ignored, and how…
In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. The study was cond…
FAIR+S: A validation study of a framework for sustainable research data and software
Carl von Ossietzky Universität Oldenburg（奥尔登堡卡尔·冯·奥西茨基大学）；OFFIS – Institute for Information Technology（OFFIS信息技术研究所）；Wilhelm Büchner Hochschule（威廉·比希纳应用科学大学）
Ethical bias in machine learning models has become a matter of concern in the software engineering community. Most of the prior software engineering works concentrated on finding ethical bias in models rather than fixing it. After finding bias, the next step is mitigation. Prior researchers mainly tried to use supervised approaches to achieve fairness. However, in the real world, getting data with trustworthy ground truth is challenging and also ground truth can contain human bias. Semi-supervised learning is a machine learning technique where, incrementally, labeled data is used to generate p…
Modern society is diverse, multicultural, and multifaceted. Because of these characteristics, we are currently observing an increase in the debates about equity, diversity, and inclusion in different areas, especially because several groups of individuals are underrepresented in many environments. In computer science and software engineering, it seems counter-intuitive that these areas, which are responsible for creating technological solutions and systems for billions of users around the world, do not reflect the diversity of the society to which it serves. In trying to solve this diversity c…
This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics inclu…
This paper describes the development of iEnvironment, an open science software platform that supports monitoring and modeling of aspects of surface water. The platform supports science and engineering research, especially in the context of the creation, sharing, analysis and maintenance of big and open data. In this era of big data, iEnvironment facilitates access to open data resources and research collaboration among science and research disciplines supported by computer scientists and software developers.
There has been growing interest within the computational science and engineering (CSE) community in engaging with software engineering research -- the systematic study of software systems and their development, operation, and maintenance -- to solve challenges in scientific software development. Historically, there has been little interaction between scientific computing and the field, which has held back progress. With the ranks of scientific software teams expanding to include software engineering researchers and practitioners, we can work to build bridges to software science and reap the re…
This report records and discusses the Fourth Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE4). The report includes a description of the keynote presentation of the workshop, the mission and vision statements that were drafted at the workshop and finalized shortly after it, a set of idea papers, position papers, experience papers, demos, and lightning talks, and a panel discussion. The main part of the report covers the set of working groups that formed during the meeting, and for each, discusses the participants, the objective and goal, and how the objective can…
This technical report records and discusses the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2). The report includes a description of the alternative, experimental submission and review process, two workshop keynote presentations, a series of lightning talks, a discussion on sustainability, and five discussions from the topic areas of exploring sustainability; software development experiences; credit & incentives; reproducibility & reuse & sharing; and code testing & code review. For each topic, the report includes a list of tangible actions that were pr…
Sensor Management System (SMS): Open-source software for FAIR sensor metadata management in Earth system sciences
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；GFZ Helmholtz Centre for Geosciences（德国地球科学研究中心）；Helmholtz Centre for Environmental Research (UFZ)（亥姆霍兹环境研究中心）；German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig（德国综合生物多样性研究哈雷-耶拿-莱比锡中心）；Forschungszentrum Jülich (FZJ)（于利希研究中心）
University of Wuppertal（伍珀塔尔大学）；FIZ Karlsruhe - Leibniz Institute for Information Infrastructure（FIZ卡尔斯鲁厄——莱布尼茨信息基础设施研究所）；National Institute of Informatics（国立信息学研究所）
Small to medium-scale data science experiments often rely on research software developed ad-hoc by individual scientists or small teams. Often there is no time to make the research software fast, reusable, and open access. The consequence is twofold. First, subsequent researchers must spend significant work hours building upon the proposed hypotheses or experimental framework. In the worst case, others cannot reproduce the experiment and reuse the findings for subsequent research. Second, suppose the ad-hoc research software fails during often long-running computationally expensive experiments…
We live in exceptional times in which the entire world is witnessing the exponential spread of a pandemic, which requires to adopt new habits of mind and behaviors. In this paper, I introduce the term exponential competence, which encompasses these cognitive and social skills, and describe a course for computer science and software engineering students in which emphasis is placed on exponential competence. I argue that exponential competence is especially important for computer science and software engineering students, since many of them will, most likely, be required to deal with exponential…
