使用变形关系验证和增强Artcode分类
Using Metamorphic Relations to Verify and Enhance Artcode Classification
摘要：
软件测试经常在不可能或不切实际地确定被测软件（SUT）的行为或输出的正确性的情况下受到阻碍，这种情况被称为Oracle问题。面临预言问题的一个领域的例子是自动图像分类，使用机器学习将输入图像分类为一组预定义类别中的一个。一种解释oracle问题的软件测试方法是变形测试（MT）。传统的软件测试只检查单个测试用例的正确性，而MT则检查多个测试用例的执行及其输出之间的关系。这些关系被称为变形关系（MR）：如果发现MR被违反，则SUT中必须存在故障。本文研究了包含视觉隐藏标记（称为Artcodes）的图像分类问题，并应用MT来验证和增强训练的分类器。本文进一步研究了两个MR，分离和闭塞，并报告他们的能力，在验证图像分类，使用单向方差分析（ANOVA）结合其他三个统计分析方法：t检验（不等方差），Kruskal-Wallis检验，和Dunnett检验。除了我们以前研究的分类器，使用随机森林，我们引入了一个新的分类器，使用支持向量机，并提出其MR增强版本。实验结果表明，增强分类器比非增强分类器具有更好的性能。本文还分析了如何获得增强的性能。
英文摘要：
Software testing is often hindered where it is impossible or impractical to determine the correctness of the behaviour or output of the software under test (SUT), a situation known as the oracle problem. An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a set of predefined classes. An approach to software testing that alleviates the oracle problem is metamorphic testing (MT). While traditional software testing examines the correctness of individual test cases, MT instead examines the relations amongst multiple executions of test cases and their outputs. These relations are called metamorphic relations (MRs): if an MR is found to be violated, then a fault must exist in the SUT. This paper examines the problem of classifying images containing visually hidden markers called Artcodes, and applies MT to verify and enhance the trained classifiers. This paper further examines two MRs, Separation and Occlusion, and reports on their capability in verifying the image classification using one-way analysis of variance (ANOVA) in conjunction with three other statistical analysis methods: t-test (for unequal variances), Kruskal-Wallis test, and Dunnett's test. In addition to our previously-studied classifier, that used Random Forests, we introduce a new classifier that uses a support vector machine, and present its MR-augmented version. Experimental evaluations across a number of performance metrics show that the augmented classifiers can achieve better performance than non-augmented classifiers. This paper also analyses how the enhanced performance is obtained.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
