arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.07919cs.CL
收藏

基于语义感知的对比学习实现更精确的语义解析

Semantic-aware Contrastive Learning for More Accurate Semantic Parsing

Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所)

Shan Wu, Chunlei Xin, Bo Chen, Xianpei Han, Le Sun

摘要:

由于语义表示是表达细粒度序列级语义的详细而准确的注释,因此通常难以通过自回归方式的最大似然估计(MLE)来训练区分性语义解析器。在本文中,我们提出了一个语义感知的对比学习算法,它可以学习区分细粒度的意义表示,并考虑到整体的序列级语义。具体而言,提出了一种多层次在线抽样算法,以采样混淆和多样化的实例。设计了三个语义感知相似度函数,以准确地测量整体意义表示之间的距离。并提出了一种排序对比损失,将语义相同的实例的表示拉到一起,将否定的实例推到一边。在两个标准数据集上的实验表明,我们的方法在MLE基线上取得了显着的改进,并通过简单地将语义感知的对比学习应用于vanilla Seq2Seq模型,获得了最先进的性能。

英文摘要:

Since the meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an autoregressive fashion. In this paper, we propose a semantic-aware contrastive learning algorithm, which can learn to distinguish fine-grained meaning representations and take the overall sequence-level semantic into consideration. Specifically, a multi-level online sampling algorithm is proposed to sample confusing and diverse instances. Three semantic-aware similarity functions are designed to accurately measure the distance between meaning representations as a whole. And a ranked contrastive loss is proposed to pull the representations of the semantic-identical instances together and push negative instances away. Experiments on two standard datasets show that our approach achieves significant improvements over MLE baselines and gets state-of-the-art performances by simply applying semantic-aware contrastive learning on a vanilla Seq2Seq model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑