arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.04792cs.SE
收藏

利用语言模型增强自然语言需求的完备性

Using Language Models for Enhancing the Completeness of Natural-language Requirements

Dipeeka Luitel, Shabnam Hassani, Mehrdad Sabetzadeh

摘要:

[上下文和动机]自然语言需求的不完整性是一个具有挑战性的问题。[问题/问题]检测需求中不完整性的一种常见技术是根据外部来源检查需求。随着语言模型(如BERT)的出现,一个有趣的问题是语言模型是否是发现需求中潜在不完整性的有用外部资源。[主要思想/结果]我们屏蔽了需求中的单词,并让BERT的屏蔽语言模型(MLM)生成用于填充屏蔽插槽的上下文预测。我们模拟的不完整性扣留内容的要求和措施BERT的能力,以预测术语,是目前在扣留的内容,但没有在内容披露给BERT。[贡献] BERT可以配置为每个掩码生成多个预测。我们的第一个贡献是确定每个掩码有多少预测是有效发现需求中的遗漏和预测中的噪声水平之间的最佳权衡。我们的第二个贡献是设计一个基于机器学习的过滤器,对BERT的预测进行后处理,以进一步降低噪声。我们从PURE数据集[1]中提取了40多个需求规范,对我们的解决方案进行了经验评估。我们的研究结果表明:(1)BERT的预测在精确定位需求中缺失的术语方面非常有效,(2)我们的过滤器可以大大减少预测中的噪声,从而使BERT成为提高需求完整性的更有说服力的辅助工具。

英文摘要:

[Context and motivation] Incompleteness in natural-language requirements is a challenging problem. [Question/problem] A common technique for detecting incompleteness in requirements is checking the requirements against external sources. With the emergence of language models such as BERT, an interesting question is whether language models are useful external sources for finding potential incompleteness in requirements. [Principal ideas/results] We mask words in requirements and have BERT's masked language model (MLM) generate contextualized predictions for filling the masked slots. We simulate incompleteness by withholding content from requirements and measure BERT's ability to predict terminology that is present in the withheld content but absent in the content disclosed to BERT. [Contribution] BERT can be configured to generate multiple predictions per mask. Our first contribution is to determine how many predictions per mask is an optimal trade-off between effectively discovering omissions in requirements and the level of noise in the predictions. Our second contribution is devising a machine learning-based filter that post-processes predictions made by BERT to further reduce noise. We empirically evaluate our solution over 40 requirements specifications drawn from the PURE dataset [1]. Our results indicate that: (1) predictions made by BERT are highly effective at pinpointing terminology that is missing from requirements, and (2) our filter can substantially reduce noise from the predictions, thus making BERT a more compelling aid for improving completeness in requirements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑