基于LoRA的大型语言模型微调在自动测试用例生成中的实证研究
An empirical study of LoRA-based fine-tuning of large language models for automated test case generation
AI总结：
本文通过实证研究探讨LoRA微调在基于需求的测试用例生成中的效果，评估了多种LLM家族，并发现LoRA微调显著提升了开源模型性能，且开源模型在微调后可与专有模型性能相当。
摘要：
从自然语言需求生成自动测试用例仍然是软件工程中的挑战，由于需求的模糊性和生成结构化可执行测试工件的需求。最近的LLM进展显示了在解决此任务中的潜力；然而，其有效性取决于任务特定的适应和高效的微调策略。本文提出了一个全面的实证研究，探讨参数高效微调（特别是LoRA）在需求驱动的测试用例生成中的应用。我们评估了多种LLM家族，包括开源和专有模型，在统一的实验流程下。研究系统地探讨了LoRA关键超参数（如秩、缩放因子和dropout）对下游性能的影响。我们提出基于GPT-4o的自动化评估框架，评估生成的测试用例在九个质量维度上的表现。实验结果表明，基于LoRA的微调显著提升了所有开源模型的性能，其中Minstral-8B在其中表现最佳。此外，我们展示了一个微调后的8B开源模型可以达到与预微调的GPT-4.1模型相当的性能，突显了参数高效适应的有效性。虽然GPT-4.1模型在总体性能上最高，但微调后专有与开源模型之间的性能差距显著缩小。这些发现为自动测试生成中的模型选择、微调策略和评估方法提供了重要见解。特别是，它们证明了在结合良好设计的微调方法时，成本高效、本地可部署的开源模型可以作为专有系统的可行替代品。
英文摘要：
Automated test case generation from natural language requirements remains a challenging problem in software engineering due to the ambiguity of requirements and the need to produce structured, executable test artifacts. Recent advances in LLMs have shown promise in addressing this task; however, their effectiveness depends on task-specific adaptation and efficient fine-tuning strategies. In this paper, we present a comprehensive empirical study on the use of parameter-efficient fine-tuning, specifically LoRA, for requirement-based test case generation. We evaluate multiple LLM families, including open-source and proprietary models, under a unified experimental pipeline. The study systematically explores the impact of key LoRA hyperparameters, including rank, scaling factor, and dropout, on downstream performance. We propose an automated evaluation framework based on GPT-4o, which assesses generated test cases across nine quality dimensions. Experimental results demonstrate that LoRA-based fine-tuning significantly improves the performance of all open-source models, with Ministral-8B achieving the best results among them. Furthermore, we show that a fine-tuned 8B open-source model can achieve performance comparable to pre-fine-tuned GPT-4.1 models, highlighting the effectiveness of parameter-efficient adaptation. While GPT-4.1 models achieve the highest overall performance, the performance gap between proprietary and open-source models is substantially reduced after fine-tuning. These findings provide important insights into model selection, fine-tuning strategies, and evaluation methods for automated test generation. In particular, they demonstrate that cost-efficient, locally deployable open-source models can serve as viable alternatives to proprietary systems when combined with well-designed fine-tuning approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
