DeepQuali: 大型语言模型在评估用户故事质量研究中的初步结果
DeepQuali: Initial results of a study on the use of large language models for assessing the quality of user stories
AI总结:
DeepQuali利用大型语言模型评估用户故事质量,通过对比专家判断发现其在整体评估上有效,但细节评分存在分歧,且需进一步融入工作流程。
摘要:
生成式人工智能(GAI),特别是大型语言模型(LLMs),在软件工程中日益被用于编码任务。然而,需求工程——特别是需求验证——在GAI应用方面却有限。当前使用GAI进行需求工程的焦点在于获取、转换和分类需求,而不是质量评估。我们提出了基于LLM(GPT-4o)的
英文摘要:
Generative artificial intelligence (GAI), specifically large language models (LLMs), are increasingly used in software engineering, mainly for coding tasks. However, requirements engineering - particularly requirements validation - has seen limited application of GAI. The current focus of using GAI for requirements is on eliciting, transforming, and classifying requirements, not on quality assessment. We propose and evaluate the LLM-based (GPT-4o) approach "DeepQuali", for assessing and improving requirements quality in agile software development. We applied it to projects in two small companies, where we compared LLM-based quality assessments with expert judgments. Experts also participated in walkthroughs of the solution, provided feedback, and rated their acceptance of the approach. Experts largely agreed with the LLM's quality assessments, especially regarding overall ratings and explanations. However, they did not always agree with the other experts on detailed ratings, suggesting that expertise and experience may influence judgments. Experts recognized the usefulness of the approach but criticized the lack of integration into their workflow. LLMs show potential in supporting software engineers with the quality assessment and improvement of requirements. The explicit use of quality models and explanatory feedback increases acceptance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。