arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.07348cs.SE
收藏

用于监督分类器测试的变形关系的故障检测有效性

Fault Detection Effectiveness of Metamorphic Relations Developed for Testing Supervised Classifiers

Prashanta Saha, Upulee Kanewala

摘要:

在机器学习中,监督分类器用于通过使用标记数据推断预测函数来获得对未标记数据的预测。监督分类器广泛应用于计算生物学、计算物理学和医疗保健等领域,以做出关键决策。然而,由于期望的答案是未知的,因此通常很难测试监督分类器。这通常被称为\emph{oracle problem},变形测试(MT)已被用于测试这样的程序。在机器翻译中,变形关系(MR)是从被测软件(SUT)的内在特征发展而来的。这些MR用于生成测试数据,并在不存在测试预言的情况下验证测试结果的正确性。MT的有效性在很大程度上取决于用于测试的MR。在本文中,我们已经进行了广泛的实证研究,以评估故障检测的有效性MR已被用于多个以前的研究,以测试监督分类。我们的研究共使用了709个可达突变体产生的多个突变引擎,并使用具有不同特征的数据集来测试SUT。我们的研究结果表明,只有14.8%的这些突变体被检测到使用MR和故障检测的有效性,这些MR不规模与突变体的数量增加时,在以前的研究中报道的相比。

英文摘要:

In machine learning, supervised classifiers are used to obtain predictions for unlabeled data by inferring prediction functions using labeled data. Supervised classifiers are widely applied in domains such as computational biology, computational physics and healthcare to make critical decisions. However, it is often hard to test supervised classifiers since the expected answers are unknown. This is commonly known as the \emph{oracle problem} and metamorphic testing (MT) has been used to test such programs. In MT, metamorphic relations (MRs) are developed from intrinsic characteristics of the software under test (SUT). These MRs are used to generate test data and to verify the correctness of the test results without the presence of a test oracle. Effectiveness of MT heavily depends on the MRs used for testing. In this paper we have conducted an extensive empirical study to evaluate the fault detection effectiveness of MRs that have been used in multiple previous studies to test supervised classifiers. Our study uses a total of 709 reachable mutants generated by multiple mutation engines and uses data sets with varying characteristics to test the SUT. Our results reveal that only 14.8\% of these mutants are detected using the MRs and that the fault detection effectiveness of these MRs do not scale with the increased number of mutants when compared to what was reported in previous studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑