在实践中引入自动测试用例生成:一个成功的故事和经验教训
On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned
摘要:
自动化的水平和质量极大地影响着软件测试活动,决定了测试过程的成本和有效性,并在很大程度上影响着最终产品的质量。虽然在工业实践中自动化许多测试活动(包括管理质量过程,执行大型测试套件,管理回归测试套件)的成本和好处是很好的理解和记录,在工业实践中自动生成系统测试套件的好处和障碍还没有很好的报告,尽管最近的自动化测试用例生成工具的进展。用于自动生成测试用例的专有工具在大型软件组织中正在成为常见的实践,并且商业工具也可用于某些应用领域和测试级别。然而,在中小型软件公司中,生成系统测试用例在很大程度上仍然是一种手工的、低效的和临时的活动。本文报告了我们的经验,介绍了自动生成系统测试套件的技术在一个中型公司。我们描述的技术和组织的障碍,我们在公司的开发过程中引入自动测试用例生成时所面临的,并提出了解决方案,我们成功地经历了在这种情况下。特别是,本文讨论了自动生成测试用例的问题,通过参考定制的ERP应用程序,中型公司开发的第三方跨国公司,并提出了ABT2.0,测试用例生成器,我们开发的剪裁ABT,一个研究国家的最先进的GUI测试生成器,他们的工业环境。本文介绍了ABT2.0的新功能,并讨论了这些新功能如何解决我们面临的问题。
英文摘要:
The level and quality of automation dramatically affects software testing activities, determines costs and effectiveness of the testing process, and largely impacts on the quality of the final product. While costs and benefits of automating many testing activities in industrial practice (including managing the quality process, executing large test suites, and managing regression test suites) are well understood and documented, the benefits and obstacles of automatically generating system test suites in industrial practice are not well reported yet, despite the recent progresses of automated test case generation tools. Proprietary tools for automatically generating test cases are becoming common practice in large software organisations, and commercial tools are becoming available for some application domains and testing levels. However, generating system test cases in small and medium-size software companies is still largely a manual, inefficient and ad-hoc activity. This paper reports our experience in introducing techniques for automatically generating system test suites in a medium-size company. We describe the technical and organisational obstacles that we faced when introducing automatic test case generation in the development process of the company, and present the solutions that we successfully experienced in that context. In particular, the paper discusses the problems of automating the generation of test cases by referring to a customised ERP application that the medium-size company developed for a third party multinational company, and presents ABT2.0, the test case generator that we developed by tailoring ABT, a research state-of-the-art GUI test generator, to their industrial environment. This paper presents the new features of ABT2.0, and discusses how these new features address the issues that we faced.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。