NILE: 形式语言自然语言描述的正式化
NILE: Formalizing Natural-Language Descriptions of Formal Languages
AI总结:
NILE通过形式化自然语言描述形式语言,利用LLM生成语法接近的Nile表达式,实现对不准确描述的自动解释。
摘要:
本文探讨了如何将形式语言的自然语言描述与形式表示进行比较,以及如何解释语义差异。这源于教育场景,其中学习者用自然语言描述形式语言(例如通过有限状态自动机、正则表达式、下推自动机、上下文无关文法或集合符号呈现),而教育支持系统需要(1)判断自然语言描述是否准确描述形式语言,以及(2)提供解释为何描述不准确。为了解决这个问题,我们引入了一个形式语言的表示语言Nile,其设计使得Nile表达式可以反映自然语言描述的形式结构。Nile具有足够的表达能力,可以涵盖广泛的形式语言,包括所有正则语言和通常用于教育场景的上下文无关语言片段。生成与自然语言描述语法接近的Nile表达式则允许算法提供不准确描述的解释。在教育数据集上的实验中,我们展示了LLM可以将自然语言描述转换为等效且语法接近的Nile表达式,具有高准确性——允许算法提供不正确自然语言描述的解释。我们的实验还显示,虽然自然语言描述也可以转换为正则表达式(但不转换为上下文无关文法),但表达式往往不语法接近,因此不适用于提供解释。
英文摘要:
This paper explores how natural-language descriptions of formal languages can be compared to their formal representations and how semantic differences can be explained. This is motivated from educational scenarios where learners describe a formal language (presented, e.g., by a finite state automaton, regular expression, pushdown automaton, context-free grammar or in set notation) in natural language, and an educational support system has to (1) judge whether the natural-language description accurately describes the formal language, and to (2) provide explanations why descriptions are not accurate. To address this question, we introduce a representation language for formal languages, Nile, which is designed so that Nile expressions can mirror the syntactic structure of natural-language descriptions of formal languages. Nile is sufficiently expressive to cover a broad variety of formal languages, including all regular languages and fragments of context-free languages typically used in educational contexts. Generating Nile expressions that are syntactically close to natural-language descriptions then allows to provide explanations for inaccuracies in the descriptions algorithmically. In experiments on an educational data set, we show that LLMs can translate natural-language descriptions into equivalent, syntactically close Nile expressions with high accuracy - allowing to algorithmically provide explanations for incorrect natural-language descriptions. Our experiments also show that while natural-language descriptions can also be translated into regular expressions (but not context-free grammars), the expressions are often not syntactically close and thus not suitable for providing explanations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。