arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.06676cs.SEcs.DL
收藏

减少软件工程中系统评审的工作量

Reducing the Effort for Systematic Reviews in Software Engineering

Francesco Osborne, Henry Muccini, Patricia Lago, Enrico Motta

摘要:

上下文系统性综述(SR)是从多个来源的相关研究的识别和分析中收集和综合证据的手段。为了实现这一目标,他们使用了一种定义明确的方法,旨在减轻偏倚风险,并确保后续更新的可重复性。然而,工作人员代表需要付出很大的努力。目标.本文的目标是介绍一种新的方法,减少人工繁琐的任务,同时利用人类的专业知识提供的价值参与SR。法从目前的方法SR,我们取代了关键词和数据提取的步骤,自动生成领域本体和分类的主要研究的方法。该方法已应用于软件体系结构的软件工程子领域,并由人工注释者进行评估。结果其结果是一种新的专家驱动的自动方法,EDAM,协助研究人员在执行SR。EDAM将本体学习技术和语义技术与人在回路相结合。第一个(由于自动化)促进了研究的可扩展性,客观性,再现性和粒度;第二个允许定制手头研究的特定重点和领域专家的知识重用。我们评估了EDAM在软件架构领域对六位资深研究人员。结果发现,资深研究人员在论文分类方面的表现与EDAM没有统计学上的显著差异。结论.由于SR中创造性较低的步骤的自动化,我们的方法允许研究人员跳过关键词和手动分类主要研究的繁琐任务,从而为分析和讨论腾出精力。

英文摘要:

Context. Systematic Reviews (SRs) are means for collecting and synthesizing evidence from the identification and analysis of relevant studies from multiple sources. To this aim, they use a well-defined methodology meant to mitigate the risks of biases and ensure repeatability for later updates. SRs, however, involve significant effort. Goal. The goal of this paper is to introduce a novel methodology that reduces the amount of manual tedious tasks involved in SRs while taking advantage of the value provided by human expertise. Method. Starting from current methodologies for SRs, we replaced the steps of keywording and data extraction with an automatic methodology for generating a domain ontology and classifying the primary studies. This methodology has been applied in the Software Engineering sub-area of Software Architecture and evaluated by human annotators. Results. The result is a novel Expert-Driven Automatic Methodology, EDAM, for assisting researchers in performing SRs. EDAM combines ontology-learning techniques and semantic technologies with the human-in-the-loop. The first (thanks to automation) fosters scalability, objectivity, reproducibility and granularity of the studies; the second allows tailoring to the specific focus of the study at hand and knowledge reuse from domain experts. We evaluated EDAM on the field of Software Architecture against six senior researchers. As a result, we found that the performance of the senior researchers in classifying papers was not statistically significantly different from EDAM. Conclusions. Thanks to automation of the less-creative steps in SRs, our methodology allows researchers to skip the tedious tasks of keywording and manually classifying primary studies, thus freeing effort for the analysis and the discussion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑