基于人工智能的问答辅助自然语言需求分析
AI-based Question Answering Assistance for Analyzing Natural-language Requirements
摘要:
由于需求是用自然语言(NL)预先编写的,因此容易出现各种缺陷,例如,不一致和不完整。因此,要求经常受到质量保证过程的制约。这些过程在完全手动执行时是乏味的,并且可能由于时间和预算压力而进一步忽视重要的质量问题。在本文中,我们提出了QAssist -一个问答(QA)的方法,提供自动化的援助,利益相关者,包括需求工程师,在分析NL的要求。在各种质量保证场景中,提出问题并获得即时答案是有益的,例如,不完全性检测由于搜索答案的范围可能超出给定的需求规范,因此自动搜索与需求相关的问题具有挑战性。为此,QAssist提供了对挖掘外部领域知识资源的支持。我们的工作是第一个倡议,汇集QA和外部领域的知识,以解决需求工程的挑战之一。我们评估QAssist的数据集,涵盖三个应用领域,共包含387个问答对。我们实验国家的最先进的QA方法,主要是基于最近的大规模语言模型。在我们的实证研究中,QAssist本地化的问题的答案,以三个段落内的需求规格和外部领域的知识资源,平均召回率分别为90.1%和96.5%。QAssist以84.2%的平均准确率提取所提出问题的实际答案。保留字:自然语言需求,问答(QA),语言模型,自然语言处理(NLP),自然语言生成(NLG),BERT,T5。
英文摘要:
By virtue of being prevalently written in natural language (NL), requirements are prone to various defects, e.g., inconsistency and incompleteness. As such, requirements are frequently subject to quality assurance processes. These processes, when carried out entirely manually, are tedious and may further overlook important quality issues due to time and budget pressures. In this paper, we propose QAssist -- a question-answering (QA) approach that provides automated assistance to stakeholders, including requirements engineers, during the analysis of NL requirements. Posing a question and getting an instant answer is beneficial in various quality-assurance scenarios, e.g., incompleteness detection. Answering requirements-related questions automatically is challenging since the scope of the search for answers can go beyond the given requirements specification. To that end, QAssist provides support for mining external domain-knowledge resources. Our work is one of the first initiatives to bring together QA and external domain knowledge for addressing requirements engineering challenges. We evaluate QAssist on a dataset covering three application domains and containing a total of 387 question-answer pairs. We experiment with state-of-the-art QA methods, based primarily on recent large-scale language models. In our empirical study, QAssist localizes the answer to a question to three passages within the requirements specification and within the external domain-knowledge resource with an average recall of 90.1% and 96.5%, respectively. QAssist extracts the actual answer to the posed question with an average accuracy of 84.2%. Keywords: Natural-language Requirements, Question Answering (QA), Language Models, Natural Language Processing (NLP), Natural Language Generation (NLG), BERT, T5.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。