面向测试骨架生成的大语言模型对比评估
Comparative Evaluation of Large Language Models for Test-Skeleton Generation
AI总结：
本研究对比评估四款大语言模型生成测试骨架的能力，通过静态分析与专家评审发现不同模型表现存在差异，提示词设计与上下文输入是质量关键因素。
摘要：
本文探索了大语言模型（Large Language Models, LLMs）在测试骨架自动化生成中的应用——测试骨架是勾勒单元测试覆盖范围、无需实现完整测试逻辑的结构模板。测试骨架在测试驱动开发（test-driven development, TDD）中尤为重要，可为系统化验证提供早期框架。传统测试骨架需手动编写，耗时且易出错，在教育场景或大规模开发环境中更是如此。我们评估了GPT-4、DeepSeek-Chat、Llama4-Maverick和Gemma2-9B四款LLM，测试它们为某大学软件工程课程开发的真实Ruby类生成RSpec骨架的能力。我们通过静态分析和盲态专家评审评估各模型的输出，衡量其结构正确性、清晰度、可维护性及对测试最佳实践的符合度。研究揭示了不同模型在解读代码结构与测试规范方面的关键差异，为使用LLM实现自动化测试搭建的实际挑战提供了洞见。结果显示，DeepSeek生成的骨架可维护性最高、结构最完善，而GPT-4的输出更完整但规范一致性较差。研究还发现提示词设计和上下文输入是影响输出质量的关键因素。
英文摘要：
This paper explores the use of Large Language Models (LLMs) to automate the generation of test skeletons -- structural templates that outline unit test coverage without implementing full test logic. Test skeletons are especially important in test-driven development (TDD), where they provide an early framework for systematic verification. Traditionally authored manually, their creation can be time-consuming and error-prone, particularly in educational or large-scale development settings. We evaluate four LLMs -- GPT-4, DeepSeek-Chat, Llama4-Maverick, and Gemma2-9B -- on their ability to generate RSpec skeletons for a real-world Ruby class developed in a university software engineering course. Each model's output is assessed using static analysis and a blind expert review to measure structural correctness, clarity, maintainability, and conformance to testing best practices. The study reveals key differences in how models interpret code structure and testing conventions, offering insights into the practical challenges of using LLMs for automated test scaffolding. Our results show that DeepSeek generated the most maintainable and well-structured skeletons, while GPT-4 produced more complete but conventionally inconsistent output. The study reveals prompt design and contextual input as key quality factors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
