FeedbackLLM: 基于元数据的多智能体语言无关测试用例生成器与演进提示和覆盖反馈
FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback
AI总结:
FeedbackLLM通过双阶段方法生成测试用例,利用元数据反馈提升覆盖率,有效解决传统方法的计算开销和可扩展性问题。
摘要:
传统测试用例生成方法常需人工干预且计算开销大,难以扩展至复杂软件系统。本文提出FeedbackLLM,基于紧密耦合的双阶段方法,通过解析源代码提取输入约束生成测试用例,并利用两个专用LLM反馈代理评估质量:Line Feedback Agent提取未执行行的元数据,Branch Feedback Agent提取未执行分支条件的元数据。该代理在双阶段过程中协同工作,重复k次迭代,并引入冗余防止缓存避免重复API请求。在C和Python标准基准程序上评估,FeedbackLLM在执行时间线性扩展的同时,展现出更高的覆盖率。
英文摘要:
Traditional approaches to test case generation often involve manual effort and incur significant computational overhead. Additionally, these approaches are not scalable, and hence, unsuitable for complex software systems. Recently, Large Language Models (LLMs) have been applied to software testing. However, single-shot prompt engineering-based approaches tend to hallucinate and generate redundant test cases, resulting in fewer branches. To handle the above-mentioned limitations, in this paper, we propose FeedbackLLM, a novel automated language-agnostic test case generation framework based on tightly coupled two-stage approach. In the first stage, FeedbackLLM extracts the input constraints by parsing source code and generates the possible test cases. The quality of the test cases is evaluated in the second stage by the following two specialized LLM feedback agents: (i) Line Feedback Agent: extracts the metadata related to missed line executions and (ii) Branch Feedback Agent: extracts the metadata of the unexecuted branch conditions. The above agents operate in a two-stage process, communicating in tandem, and this procedure is repeated for k-steps. Further, we also introduced a redundancy prevention cache to avoid duplicate API requests and avoid unnecessary execution cycles. The performance of the proposed architecture is evaluated on the standard benchmark programs related to C and Python programs. FeedbackLLM demonstrated more line and branch coverage than baseline tools while scaling linearly in execution time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。