Background: As Machine Learning (ML) advances rapidly in many fields, it is being adopted by academics and businesses alike. However, ML has a number of different challenges in terms of maintenance not found in traditional software projects. Identifying what causes these maintainability challenges can help mitigate them early and continue delivering value in the long run without degrading ML performance. Aim: This study aims to identify and synthesise the maintainability challenges in different stages of the ML workflow and understand how these stages are interdependent and impact each other's maintainability. Method: Using a systematic literature review, we screened more than 13000 papers, then selected and qualitatively analysed 56 of them. Results: (i) a catalogue of maintainability challenges in different stages of Data Engineering, Model Engineering workflows and the current challenges when building ML systems are discussed; (ii) a map of 13 maintainability challenges to different interdependent stages of ML that impact the overall workflow; (iii) Provided insights to developers of ML tools and researchers. Conclusions: In this study, practitioners and organisations will learn about maintainability challenges and their impact at different stages of ML workflow. This will enable them to avoid pitfalls and help to build a maintainable ML system. The implications and challenges will also serve as a basis for future research to strengthen our understanding of the ML system's maintainability.
The software is changing rapidly with the invention of advanced technologies and methodologies. The ability to rapidly and successfully upgrade software in response to changing business requirements is more vital than ever. For the long-term management of software products, measuring software maintainability is crucial. The use of soft computing techniques for software maintainability prediction has shown immense promise in software maintenance process by providing accurate prediction of software maintainability. To better understand the role of soft computing techniques for software maintaina…
Towards Evidence-Based Ontology for Supporting Systematic Literature Review
The Lab for Internet Software Technologies Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所互联网软件技术实验室）；Graduate University of Chinese Academy of Sciences（中国科学院研究生院）；National ICT Australia, University of New South Wales（澳大利亚国家信息与通信技术研究院，新南威尔士大学）
[研究背景]：系统文献综述（Systematic Literature Review，SLR）已成为软件工程的一种重要研究方法。[Aim]本文提出了一种方法，利用经验演变的本体，以支持自动化的关键SLR活动。【方法】：首先，我们提出了一个本体，SLRONT，建立在SLR经验和最佳实践的基础上，以捕捉常见的术语和它们之间的关系在SLR过程中;第二，我们提出了一个扩展版本的SLRONT，COSONT和实例化它的知识和概念提取结构化摘要。案例研究说明了应用它来支持SLR步骤的细节。【结果】：结果表明，通过使用COSONT，我们得到了相同的结论，与纯粹的手工工作相比，但所涉及的工作量显着减少。[结论]利用本体论的方法可以有效地支持系统性文献综述的开展。
The University of Queensland（昆士兰大学）；Leipzig University（莱比锡大学）；CSIRO（联邦科学与工业研究组织）
Systematic reviews are comprehensive reviews of the literature for a highly focused research question. These reviews are often treated as the highest form of evidence in evidence-based medicine, and are the key strategy to answer research questions in the medical field. To create a high-quality systematic review, complex Boolean queries are often constructed to retrieve studies for the review topic. However, it often takes a long time for systematic review researchers to construct a high quality systematic review Boolean query, and often the resulting queries are far from effective. Poor queri…
Artificial intelligence (AI)-based models used to improve different fields including healthcare, and finance. One of the field that receive advantages of AI is automation. However, it is important to consider human factors in application of AI in automation. This paper reports on a systematic review of the published studies used to investigate the application of AI in PM. This comprehensive systematic review used ScienceDirect to identify relevant articles. Of the 422 articles found, 40 met the inclusion and exclusion criteria and were used in the review. Selected articles were classified base…
School of Computer Science, College of Science and Health, Technological University Dublin（都柏林理工大学科学与健康学院计算机科学学院）；Technological University Dublin（都柏林理工大学）
Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that has led to the development of highly accurate models but lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested. This systematic review contributes to the body of knowledge by clustering these methods with a hierarchical classification system with four main clusters: review articles, theories and notions, methods and their ev…
University of Queensland（昆士兰大学）；University of Newcastle（纽卡斯尔大学）；International Computer Science Institute（国际计算机科学研究所）
Reinforcement learning is important part of artificial intelligence. In this paper, we review model-free reinforcement learning that utilizes the average reward optimality criterion in the infinite horizon setting. Motivated by the solo survey by Mahadevan (1996a), we provide an updated review of work in this area and extend it to cover policy-iteration and function approximation methods (in addition to the value-iteration and tabular counterparts). We present a comprehensive literature mapping. We also identify and discuss opportunities for future work.
Artificial intelligence (AI) algorithms have emerged in the educational domain as a tool to make learning more efficient. Different applications for mastering particular skills, learning new languages, and tracking their progress are used by children. What is the impact on children from using this smart technology? We conducted a systematic review to understand the state of the art. We explored the literature in several sub-disciplines: wearables, child psychology, AI and education, school surveillance, and accountability. Our review identified the need for more research for each established t…
University of Queensland（昆士兰大学）；Leipzig University（莱比锡大学）；CSIRO（联邦科学与工业研究组织）
Medical systematic reviews typically require assessing all the documents retrieved by a search. The reason is two-fold: the task aims for ``total recall''; and documents retrieved using Boolean search are an unordered set, and thus it is unclear how an assessor could examine only a subset. Screening prioritisation is the process of ranking the (unordered) set of retrieved documents, allowing assessors to begin the downstream processes of the systematic review creation earlier, leading to earlier completion of the review, or even avoiding screening documents ranked least relevant. Screening pri…
Cybersecurity vendors consistently apply AI (Artificial Intelligence) to their solutions and many cybersecurity domains can benefit from AI technology. However, black-box AI techniques present some difficulties in comprehension and adoption by its operators, given that their decisions are not always humanly understandable (as is usually the case with deep neural networks, for example). Since it aims to make the operation of AI algorithms more interpretable for its users and developers, XAI (eXplainable Artificial Intelligence) can be used to address this issue. Through a systematic literature …
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Raytheon BBN Technologies（雷神BBN科技公司）；Army Research Laboratory（陆军研究实验室）
Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use of Deep Neural Networks for automated and objective surgical skill assessment, with a focus on kinematic data as putative markers of surgical competency. There is considerable recent interest in deep…
Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages
Trinity Centre for Global Health（三一全球健康中心）；School of Psychology（心理学院）；Trinity College Dublin（都柏林圣三一学院）；University of York（约克大学）
