arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.04864cs.LOcs.AIcs.LG
收藏

nl2spec:基于大语言模型的交互式非结构化自然语言时态逻辑转换

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Stanford University(斯坦福大学)

Matthias Cosler, Christopher Hahn, Daniel Mendoza, Frederik Schmitt, Caroline Trippel

摘要:

在执行任何验证任务时,所需系统需求的严格形式化是必不可少的。这通常限制了验证技术的应用,因为编写正式规范是一项容易出错且耗时的手动任务。为了促进这一点,我们提出了nl2spec,应用大语言模型(LLM)从非结构化自然语言中获得正式规范(在时序逻辑)的框架。特别是,我们引入了一种新的方法来检测和解决固有的模糊性的系统要求在自然语言:我们利用LLM映射子公式的形式化回相应的自然语言片段的输入。用户迭代地添加、删除和编辑这些子翻译以修改错误的形式化,这比手动重新起草整个形式化更容易。该框架对特定的应用领域是不可知的,并且可以扩展到类似的规约语言和新的神经模型。我们进行用户研究以获得具有挑战性的数据集,我们使用该数据集对翻译质量进行实验。我们提供了一个开源的实现,包括一个基于Web的前端。

英文摘要:

A rigorous formalization of desired system requirements is indispensable when performing any verification task. This often limits the application of verification techniques, as writing formal specifications is an error-prone and time-consuming manual task. To facilitate this, we present nl2spec, a framework for applying Large Language Models (LLMs) to derive formal specifications (in temporal logics) from unstructured natural language. In particular, we introduce a new methodology to detect and resolve the inherent ambiguity of system requirements in natural language: we utilize LLMs to map subformulas of the formalization back to the corresponding natural language fragments of the input. Users iteratively add, delete, and edit these sub-translations to amend erroneous formalizations, which is easier than manually redrafting the entire formalization. The framework is agnostic to specific application domains and can be extended to similar specification languages and new neural models. We perform a user study to obtain a challenging dataset, which we use to run experiments on the quality of translations. We provide an open-source implementation, including a web-based frontend.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑