利用产品文档提升大语言模型代码生成质量的研究
A Study on the Improvement of Code Generation Quality Using Large Language Models Leveraging Product Documentation
AI总结：
针对现有E2E测试生成研究不足的问题，提出利用定制提示词让LLMs基于产品文档两步生成E2E测试代码的方法，实验证明其效果优于需求规格和用户故事方案，可提升软件质量。
摘要：
大语言模型（LLMs）在系统开发中的应用研究正不断拓展，尤其在自动化代码与测试生成领域。尽管端到端（E2E）测试对保障应用质量至关重要，但现有测试生成研究多聚焦于单元测试，针对E2E测试代码的研究较为有限。本研究提出一种方法，借助定制化提示词，利用LLMs从手册、FAQ、教程等产品文档中自动生成E2E测试代码。该方法分为两步：先解读文档意图，再生成可执行测试代码。在具备认证、个人资料、讨论等6项核心功能的Web应用上开展的实验表明，基于产品文档生成的测试编译成功率高、功能覆盖度好，表现优于基于需求规格说明和用户故事生成的测试。这些发现凸显了产品文档在提升E2E测试质量、进而改善软件质量方面的潜力。
英文摘要：
Research on using Large Language Models (LLMs) in system development is expanding, especially in automated code and test generation. While E2E testing is vital for ensuring application quality, most test generation research has focused on unit tests, with limited work on E2E test code. This study proposes a method for automatically generating E2E test code from product documentation such as manuals, FAQs, and tutorials using LLMs with tailored prompts. The two step process interprets documentation intent and produces executable test code. Experiments on a web app with six key features (e.g., authentication, profile, discussion) showed that tests generated from product documentation had high compilation success and functional coverage, outperforming those based on requirement specs and user stories. These findings highlight the potential of product documentation to improve E2E test quality and, by extension, software quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
