分析SE中的实验族:系统映射研究
Analyzing Families of Experiments in SE: A Systematic Mapping Study
摘要:
背景:实验系列(即,在软件工程(SE)中,具有相同目标的实验组)正在增加。选择不合适的聚合技术来分析家庭可能会破坏他们的潜力,提供深入的见解,从实验的结果。目的:确定用于在SE家族内聚集实验结果的技术。提高对应用适当的汇总技术以在家庭内部得出可靠结论的重要性的认识。方法:我们进行了一个系统的映射研究(SMS),以确定用于分析家庭的SE实验的聚合技术。我们根据医学和药理学等成熟的实验学科概述了每种聚集技术的优点和缺点。我们提供了初步的建议,以分析和报告家庭的实验,鉴于家庭的共同限制,联合数据分析。结果如下:几种聚合技术已被用于分析SE系列实验,包括叙述合成、聚合数据(AD)、个体参与者数据(IPD)大试验或分层以及p值聚合。用于选择汇总技术的理由很少在家庭内部讨论。根据成熟实验学科的文献,通常使用不合适的聚集技术来分析实验族。结论:应改进数据分析的报告实践,以提高联合结果的可靠性和透明度。AD和IPD分层似乎适合于分析SE系列实验。
英文摘要:
Context: Families of experiments (i.e., groups of experiments with the same goal) are on the rise in Software Engineering (SE). Selecting unsuitable aggregation techniques to analyze families may undermine their potential to provide in-depth insights from experiments' results. Objectives: Identifying the techniques used to aggregate experiments' results within families in SE. Raising awareness of the importance of applying suitable aggregation techniques to reach reliable conclusions within families. Method: We conduct a systematic mapping study (SMS) to identify the aggregation techniques used to analyze families of experiments in SE. We outline the advantages and disadvantages of each aggregation technique according to mature experimental disciplines such as medicine and pharmacology. We provide preliminary recommendations to analyze and report families of experiments in view of families' common limitations with regard to joint data analysis. Results: Several aggregation techniques have been used to analyze SE families of experiments, including Narrative synthesis, Aggregated Data (AD), Individual Participant Data (IPD) mega-trial or stratified, and Aggregation of p-values. The rationale used to select aggregation techniques is rarely discussed within families. Families of experiments are commonly analyzed with unsuitable aggregation techniques according to the literature of mature experimental disciplines. Conclusion: Data analysis' reporting practices should be improved to increase the reliability and transparency of joint results. AD and IPD stratified appear to be suitable to analyze SE families of experiments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。