使用ChatGPT从场景中自动提取用例组件的分析
An Analysis of Automated Use Case Component Extraction from Scenarios using ChatGPT
AI总结:
本文研究如何利用ChatGPT等大语言模型从用户场景中自动提取用例组件,通过引入领域知识并优化提示设计,在50个标注场景上提升了提取的精确率和召回率。
摘要:
移动应用程序(apps)通常仅由少数资源有限的开发者开发,尤其是在应用开发的早期阶段。在这种环境下,许多需求获取活动(如访谈)具有挑战性,或者其优先级低于开发和发布活动。此外,在这一早期阶段,随着移动应用为在市场竞争中不断演进,需求也频繁变化。然而,随着应用开发公司逐步标准化其开发流程,他们将转向记录和分析需求。部署后的一种低成本需求来源是用户撰写的描述其如何与应用交互的场景。我们提出了一种利用大语言模型(LLMs)从用户撰写的场景中提取用例组件的方法。该方法由一系列提示组成,这些提示是为了在一个由50个场景组成、并独立标注了UC组件的地面真值数据集上提高精确率和召回率而开发的。我们的结果表明,LLMs需要额外的领域知识才能提取UC组件,并且通过改进提示以包含这些知识可以提高所提取UC组件的质量。
英文摘要:
Mobile applications (apps) are often developed by only a small number of developers with limited resources, especially in the early years of the app's development. In this setting, many requirements acquisition activities, such as interviews, are challenging or lower priority than development and release activities. Moreover, in this early period, requirements are frequently changing as mobile apps evolve to compete in the marketplace. As app development companies move to standardize their development processes, however, they will shift to documenting and analyzing requirements. One low-cost source of requirements post-deployment are user-authored scenarios describing how they interact with an app. We propose a method for extracting use case components from user-authored scenarios using large language models (LLMs). The method consists of a series of prompts that were developed to improve precision and recall on a ground truth dataset of 50 scenarios independently labeled with UC components. Our results reveal that LLMs require additional domain knowledge to extract UC components, and that refining prompts to include this knowledge improves the quality of the extracted UC components.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。