arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1706.00933cs.SE
收藏

实证软件工程研究中统计分析的演变:现状与前进步骤

Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Francisco Gomes de Oliveira Neto, Richard Torkar, Robert Feldt, Lucas Gren, Carlo A. Furia, Ziwei Huang

AI总结:

该研究通过人工审阅161篇论文并半自动分类2001—2015年5,196篇论文,分析实证软件工程统计分析实践与趋势,提出工作流概念模型并强调实际显著性报告。

摘要:

软件工程研究正在不断发展,论文越来越多地基于来自多种来源的实证数据,并使用统计检验来确定实证证据是否以及在多大程度上支持其假设。为了调查实证软件工程(ESE)中统计分析的实践和趋势,本文对来自顶级软件工程期刊的大量论文进行了综述。首先,我们人工审阅了161篇论文;在方法的第二阶段,我们对2001年至2015年间的5,196篇论文进行了更广泛的半自动分类。两个综述步骤的结果被用于:i)识别并分析ESE中的主要实践(例如使用t检验或ANOVA),以及特定统计方法使用方面的相关趋势(例如非参数检验和效应量度量);ii)开发一个统计分析工作流的概念模型,并就如何应用不同统计方法提出建议,同时提供避免陷阱的指南。最后,我们证实了现有主张,即当前ESE实践缺乏报告结果实际显著性的标准。我们说明了如何从统计分析以及从业者情境两个方面讨论实际显著性。

英文摘要:

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑