arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.00577cs.CL
收藏

基于语义分析的逻辑一致性文本生成

Logic-Consistency Text Generation from Semantic Parses

Beihang University(北京航空航天大学);University of Waterloo(滑铁卢大学);The University of Hong Kong(香港大学);Penn State University(宾夕法尼亚州立大学);University of Edinburgh(爱丁堡大学)

Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi, Tao Yu, Rui Zhang

摘要:

从语义分析生成文本是为形式化表示输入(如逻辑表单和SQL查询)生成文本描述。由于两个原因,这是具有挑战性的:(1)具有数据稀缺约束的复杂和密集的内部逻辑,(2)缺乏逻辑一致性的自动评估指标。为了解决这两个挑战,本文首先提出了SNOWBALL,一个框架的逻辑一致的文本生成的语义分析,采用迭代训练过程,通过递归增强训练集的质量控制。其次,我们提出了一种新的自动度量,BLEC,用于评估语义分析和生成的文本之间的逻辑一致性。在两个基准数据集Logic2Text和Spider上的实验结果表明,SNOWBALL框架增强了BLEC和人工评估的逻辑一致性。此外,我们的统计分析表明,BLEC是更符合逻辑的人的评价比通用的自动指标,包括BLEU,ROUGE和,BLEURT。我们的数据和代码可在https://github.com/Ciaranshu/relogic上获得。

英文摘要:

Text generation from semantic parses is to generate textual descriptions for formal representation inputs such as logic forms and SQL queries. This is challenging due to two reasons: (1) the complex and intensive inner logic with the data scarcity constraint, (2) the lack of automatic evaluation metrics for logic consistency. To address these two challenges, this paper first proposes SNOWBALL, a framework for logic consistent text generation from semantic parses that employs an iterative training procedure by recursively augmenting the training set with quality control. Second, we propose a novel automatic metric, BLEC, for evaluating the logical consistency between the semantic parses and generated texts. The experimental results on two benchmark datasets, Logic2Text and Spider, demonstrate the SNOWBALL framework enhances the logic consistency on both BLEC and human evaluation. Furthermore, our statistical analysis reveals that BLEC is more logically consistent with human evaluation than general-purpose automatic metrics including BLEU, ROUGE and, BLEURT. Our data and code are available at https://github.com/Ciaranshu/relogic.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑