探索性能度量与缺陷预测模型的成本节约潜力之间的关系
Exploring the relationship between performance metrics and cost saving potential of defect prediction models
摘要：
内容：性能指标是评估任何机器学习模型的核心组成部分，用于比较模型并估计其有用性。最近的工作开始质疑许多性能指标的有效性，为此目的，在软件缺陷预测的上下文中。目的：在这项研究中，我们探讨性能指标和成本节约的缺陷预测模型的潜力之间的关系。我们研究是否性能指标是合适的代理，以评估成本节约能力，并得出一个理论的性能指标和成本节约潜力之间的关系。方法：我们在缺陷预测实验中测量性能指标和成本节约潜力。我们使用多项logit模型、决策和随机森林来模拟指标与成本节约之间的关系。结果：我们无法找到成本节约和性能指标之间的稳定关系。我们属性的关系的缺乏，性能指标无法占的财产，一小部分非常大的软件工件的主要驱动程序的成本。结论：任何对寻找最佳预测模型感兴趣的缺陷预测研究都必须直接考虑成本节约，因为否则就无法对缺陷预测的经济效益提出合理的要求。
英文摘要：
Context: Performance metrics are a core component of the evaluation of any machine learning model and used to compare models and estimate their usefulness. Recent work started to question the validity of many performance metrics for this purpose in the context of software defect prediction. Objective: Within this study, we explore the relationship between performance metrics and the cost saving potential of defect prediction models. We study whether performance metrics are suitable proxies to evaluate the cost saving capabilities and derive a theory for the relationship between performance metrics and cost saving potential. Methods: We measure performance metrics and cost saving potential in defect prediction experiments. We use a multinomial logit model, decision, and random forest to model the relationship between the metrics and the cost savings. Results: We could not find a stable relationship between cost savings and performance metrics. We attribute the lack of the relationship to the inability of performance metrics to account for the property that a small proportion of very large software artifacts are the main driver of the costs. Conclusion: Any defect prediction study interested in finding the best prediction model, must consider cost savings directly, because no reasonable claims regarding the economic benefits of defect prediction can be made otherwise.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
