基于相似度的跨项目缺陷预测实例选择方法比较
A Comparison of Similarity Based Instance Selection Methods for Cross Project Defect Prediction
摘要：
内容：以往的研究表明，基于最近邻信息的训练数据实例选择可以通过减少训练数据集的异质性来提高跨项目缺陷预测（CPDP）的性能。然而，邻域计算在计算上是昂贵的，并且近似方法（诸如局部敏感哈希（LSH））可以与精确方法一样有效。目的：我们的目的是比较CPDP的实例选择方法，即LSH，NN过滤器和遗传实例选择（GIS）。方法：我们进行了实验，5个基地的学习者，优化他们的超参数，从PROMISE存储库的13个数据集，以比较LSH的性能与基准实例选择方法NN过滤器和GIS。结果：统计检验显示六个不同的组的F-测量性能。顶部的两组仅包含LSH和GIS基准，而底部的两组仅包含NN过滤器变体。LSH和GIS更倾向于查全率而不是查准率。事实上，对于精度性能，测试仅检测到三个显著不同的组，其中顶部组仅由NN过滤器变体组成。回想一下，识别了16个不同的组，其中前三个组仅包含LSH方法，接下来的六个中的四个仅包含GIS，并且底部的五个仅包含NN过滤器。最后，NN过滤器基准测试永远不会优于具有相同基础学习器的LSH对应测试，无论是否调优。此外，它们甚至从来不属于同一个等级组，这意味着LSH总是明显优于具有相同学习器和设置的NN过滤器。结论：性能的提高和计算开销和运行时间的减少使LSH成为一种有前途的方法。然而，LSH的性能是基于高召回率的，并且在精度被认为更重要的环境中，应该考虑NN过滤器。
英文摘要：
Context: Previous studies have shown that training data instance selection based on nearest neighborhood (NN) information can lead to better performance in cross project defect prediction (CPDP) by reducing heterogeneity in training datasets. However, neighborhood calculation is computationally expensive and approximate methods such as Locality Sensitive Hashing (LSH) can be as effective as exact methods. Aim: We aim at comparing instance selection methods for CPDP, namely LSH, NN-filter, and Genetic Instance Selection (GIS). Method: We conduct experiments with five base learners, optimizing their hyper parameters, on 13 datasets from PROMISE repository in order to compare the performance of LSH with benchmark instance selection methods NN-Filter and GIS. Results: The statistical tests show six distinct groups for F-measure performance. The top two group contains only LSH and GIS benchmarks whereas the bottom two groups contain only NN-Filter variants. LSH and GIS favor recall more than precision. In fact, for precision performance only three significantly distinct groups are detected by the tests where the top group is comprised of NN-Filter variants only. Recall wise, 16 different groups are identified where the top three groups contain only LSH methods, four of the next six are GIS only and the bottom five contain only NN-Filter. Finally, NN-Filter benchmarks never outperform the LSH counterparts with the same base learner, tuned or non-tuned. Further, they never even belong to the same rank group, meaning that LSH is always significantly better than NN-Filter with the same learner and settings. Conclusions: The increase in performance and the decrease in computational overhead and runtime make LSH a promising approach. However, the performance of LSH is based on high recall and in environments where precision is considered more important NN-Filter should be considered.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
