软件工程中聚集相关复制的比较技术
Comparing Techniques for Aggregating Interrelated Replications in Software Engineering
摘要:
背景:来自不同团体和机构的研究人员正在合作构建相互关联的复制组。采用不适当的技术来聚合相关重复的结果可能会影响联合结论的可靠性。目的:比较了在软件工程(SE)中应用于聚合相关复制结果的技术的优缺点。方法:我们进行了文献综述,以确定应用于SE中聚合相关重复结果的技术。我们分析了一组典型的相互关联的复制SE与我们确定的技术。我们检查每种技术的优点和缺点-根据成熟的实验学科,如医学-在SE背景下实现。结果如下:叙述性综合和p值汇总未利用原始数据中包含的所有信息来提供联合结论。聚合数据(AD)荟萃分析提供了结果的可视化摘要,并允许评估实验水平的主持人。个体参与者数据(IPD)元分析允许以自然单位解释结果并评估实验级和参与者级主持人。结论:原始数据中包含的所有信息都应用于提供联合结论。AD和IPD,当串联使用时,似乎适合于分析SE中相互关联的复制组。
英文摘要:
Context: Researchers from different groups and institutions are collaborating towards the construction of groups of interrelated replications. Applying unsuitable techniques to aggregate interrelated replications' results may impact the reliability of joint conclusions. Objectives: Comparing the advantages and disadvantages of the techniques applied to aggregate interrelated replications' results in Software Engineering (SE). Method: We conducted a literature review to identify the techniques applied to aggregate interrelated replications' results in SE. We analyze a prototypical group of interrelated replications in SE with the techniques that we identified. We check whether the advantages and disadvantages of each technique -according to mature experimental disciplines such as medicine- materialize in the SE context. Results: Narrative synthesis and Aggregation of p-values do not take advantage of all the information contained within the raw-data for providing joint conclusions. Aggregated Data (AD) meta-analysis provides visual summaries of results and allows assessing experiment-level moderators. Individual Participant Data (IPD) meta-analysis allows interpreting results in natural units and assessing experiment-level and participant-level moderators. Conclusion: All the information contained within the raw-data should be used to provide joint conclusions. AD and IPD, when used in tandem, seem suitable to analyze groups of interrelated replications in SE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。