MR-Scout:从现有测试用例自动合成元变换关系
MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases
AI总结:
MR-Scout通过分析开源项目中的测试用例自动合成元变换关系,提升测试覆盖率和变异评分,验证了其在自动化测试中的有效性。
摘要:
元变换测试(MT)通过定义元变换关系(MRs)来缓解Oracle问题,但设计MRs需要领域知识,限制了MT的普及。本文提出MR-Scout,从开源软件项目中的测试用例自动合成MRs。MR-Scout首先发现包含MR编码的测试用例(MTCs),然后将编码的MRs合成参数化方法(称为编码MRs),并过滤出质量低的MRs用于新测试用例生成。MR-Scout在701个开源项目中发现了超过11,000个MTCs。实验结果表明,超过97%的编码MRs在自动化测试用例生成中具有高质量,证明了MR-Scout的实用性。此外,基于编码MRs的测试有效提高了程序的测试充分性,导致行覆盖率和变异评分分别增加13.52%和9.42%。我们的定性研究显示,55.76%至76.92%的编码MRs对开发人员易于理解。
英文摘要:
Metamorphic Testing (MT) alleviates the oracle problem by defining oracles based on metamorphic relations (MRs), that govern multiple related inputs and their outputs. However, designing MRs is challenging, as it requires domain-specific knowledge. This hinders the widespread adoption of MT. We observe that developer-written test cases can embed domain knowledge that encodes MRs. Such encoded MRs could be synthesized for testing not only their original programs but also other programs that share similar functionalities. In this paper, we propose MR-Scout to automatically synthesize MRs from test cases in open-source software (OSS) projects. MR-Scout first discovers MR-encoded test cases (MTCs), and then synthesizes the encoded MRs into parameterized methods (called codified MRs), and filters out MRs that demonstrate poor quality for new test case generation. MR-Scout discovered over 11,000 MTCs from 701 OSS projects. Experimental results show that over 97% of codified MRs are of high quality for automated test case generation, demonstrating the practical applicability of MR-Scout. Furthermore, codified-MRs-based tests effectively enhance the test adequacy of programs with developer-written tests, leading to 13.52% and 9.42% increases in line coverage and mutation score, respectively. Our qualitative study shows that 55.76% to 76.92% of codified MRs are easily comprehensible for developers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。