生成具有描述性名称的REST API测试
Generating REST API Tests With Descriptive Names
AI总结:
本文提出三种确定性技术生成REST API测试的描述性名称,并通过实验验证其有效性,证明轻量级方法可作为替代LLM方法的实用方案。
摘要:
自动化测试生成已成为确保软件质量的关键技术,特别是在现代基于API的架构中。然而,自动生成的测试用例通常被分配非描述性的名称(例如test0、test1),这降低了其可读性,并阻碍了在理解和维护中的实用性。在本工作中,我们提出了三种新颖的确定性技术来生成REST API测试名称。然后,我们总共比较了八种技术,以生成由fuzzer EvoMaster自动生成的REST API测试的描述性名称,使用为9个不同的开源API生成的10个测试用例。这八种技术包括基于规则的启发式方法和基于大型语言模型(LLM)的方法。通过两次调查(涉及通过LinkedIn招募的最多39人)对这些方法的有效性进行了实证评估。我们的结果表明,基于规则的方法在确定性方法中实现了最高的清晰度评分,其性能与最先进的LLM模型如Gemini和GPT-4o相当,并显著优于GPT-3.5。为了进一步评估我们结果的实际影响,与在大众汽车集团(Volkswagen AG)积极使用EvoMaster的从业者进行了工业案例研究。然后,基于四个不同API由四个不同用户使用EvoMaster的情况,进行了开发者问卷调查,总计评估了74个测试用例。从业者反馈进一步确认,通过这种方法生成的描述性名称提高了测试套件的可读性。这些发现强调,轻量级、确定性的技术可以作为计算成本高且安全敏感的LLM方法的有效替代方案,为自动化系统级测试命名提供了一种实用的步骤,朝着更友好的开发者API测试生成迈进。
英文摘要:
Automated test generation has become a key technique for ensuring software quality, particularly in modern API-based architectures. However, automatically generated test cases are typically assigned non-descriptive names (e.g., test0, test1), which reduces their readability and hinders their usefulness during comprehension and maintenance. In this work, we present three novel deterministic techniques to generate REST API test names. We then compare eight techniques in total for generating descriptive names for REST API tests automatically produced by the fuzzer EvoMaster, using 10 test cases generated for 9 different open-source APIs. The eight techniques include rule-based heuristics and large language model (LLM)-based approaches. Their effectiveness was empirically evaluated through two surveys (involving up to 39 people recruited via LinkedIn). Our results show that a rule-based approach achieves the highest clarity ratings among deterministic methods, performs on par with state-of-the-art LLM-based models such as Gemini and GPT-4o, and significantly outperforms GPT-3.5. To further evaluate the practical impact of our results, an industrial case study was carried out with practitioners who actively use EvoMaster at Volkswagen AG. A developer questionnaire was then carried out based on the use of EvoMaster on four different APIs by four different users, for a total of 74 evaluated test cases. Feedback from practitioners further confirms that descriptive names produced by this approach improve test suite readability. These findings highlight that lightweight, deterministic techniques can serve as effective alternatives to computationally expensive and security-sensitive LLM-based approaches for automated system-level test naming, providing a practical step toward more developer-friendly API test generation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。