arXiv 2106.033762021-06-07cs.CL

A Globally Normalized Neural Model for Semantic Parsing

Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）；University of Alberta（阿尔伯塔大学）；Borealis AI（北极光人工智能研究院）

Chenyang Huang, Wei Yang, Yanshuai Cao, Osmar Zaïane, Lili Mou

英文摘要：

In this paper, we propose a globally normalized model for context-free grammar (CFG)-based semantic parsing. Instead of predicting a probability, our model predicts a real-valued score at each step and does not suffer from the label bias problem. Experiments show that our approach outperforms locally normalized models on small datasets, but it does not yield improvement on a large dataset.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Non-Projective Dependency Parsing via Latent Heads Representation (LHR)

In this paper, we introduce a novel approach based on a bidirectional recurrent autoencoder to perform globally optimized non-projective dependency parsing via semi-supervised learning. The syntactic analysis is completed at the end of the neural process that generates a Latent Heads Representation (LHR), without any algorithmic constraint and with a linear complexity. The resulting "latent syntactic structure" can be used directly in other semantic tasks. The LHR is transformed into the usual dependency tree computing a simple vectors similarity. We believe that our model has the potential to…

2018-02-06cs.CL

CRF Autoencoder for Unsupervised Dependency Parsing

School of Information Science and Technology（信息科学与技术学院）；ShanghaiTech University（上海科技大学）

Unsupervised dependency parsing, which tries to discover linguistic dependency structures from unannotated data, is a very challenging task. Almost all previous work on this task focuses on learning generative models. In this paper, we develop an unsupervised dependency parsing model based on the CRF autoencoder. The encoder part of our model is discriminative and globally normalized which allows us to use rich features as well as universal linguistic priors. We propose an exact algorithm for parsing as well as a tractable learning algorithm. We evaluated the performance of our model on eight …

2017-08-03cs.CL

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations

Bar-Ilan University（巴伊兰大学）

We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very sim…

2016-07-20cs.CL

Globally Normalized Transition-Based Neural Networks

Columbia University（哥伦比亚大学）；Google Inc（谷歌公司）

We introduce a globally normalized transition-based neural network model that achieves state-of-the-art part-of-speech tagging, dependency parsing and sentence compression results. Our model is a simple feed-forward neural network that operates on a task-specific transition system, yet achieves comparable or better accuracies than recurrent models. We discuss the importance of global as opposed to local normalization: a key insight is that the label bias problem implies that globally normalized models can be strictly more expressive than locally normalized models.

2016-06-08cs.CLcs.LGcs.NE

Global Reasoning over Database Structures for Text-to-SQL Parsing

Tel-Aviv University（特拉维夫大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）

State-of-the-art semantic parsers rely on auto-regressive decoding, emitting one symbol at a time. When tested against complex databases that are unobserved at training time (zero-shot), the parser often struggles to select the correct set of database constants in the new database, due to the local nature of decoding. In this work, we propose a semantic parser that globally reasons about the structure of the output query to make a more contextually-informed selection of database constants. We use message-passing through a graph neural network to softly select a subset of database constants for…

2019-08-29cs.CL

基于定理证明的神经语义解析评估

A Theorem-Proving-Based Evaluation of Neural Semantic Parsing

Kyoto University（京都大学）；Keio University（庆应大学）；Kikagaku Inc.（麒麟株式会社）

本文指出 Smatch 等图匹配指标仅捕捉表面重叠而非逻辑等价，通过结合自动定理证明重新评估神经语义解析器，发现图匹配表现好的模型常无法生成逻辑等价公式，归一化能提升逻辑充分性，凸显了逻辑敏感评估的必要性。

2025-10-13cs.CL

更多相关论文收起更多论文

A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese

NVIDIA（英伟达）；VinAI Research（越南人工智能研究院）

Semantic parsing is an important NLP task. However, Vietnamese is a low-resource language in this research area. In this paper, we present the first public large-scale Text-to-SQL semantic parsing dataset for Vietnamese. We extend and evaluate two strong semantic parsing baselines EditSQL (Zhang et al., 2019) and IRNet (Guo et al., 2019) on our dataset. We compare the two baselines with key configurations and find that: automatic Vietnamese word segmentation improves the parsing results of both baselines; the normalized pointwise mutual information (NPMI) score (Bouma, 2009) is useful for sche…

2020-10-05cs.AIcs.CL

Semi-supervised Autoencoding Projective Dependency Parsing

Purdue University（普渡大学）

We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the input using continuous latent variables in a sequential manner; The second model is a Globally Autoencoding Parser (GAP) encoding the input into dependency trees as latent variables, with exact inference. Both models consist of two parts: an encoder enhanced by deep neural networks (DNN) that can utilize the contextual information to encode the input into latent variables, and a decoder which is a generative model able…

2020-11-02cs.CLcs.LG

DORIC : Domain Robust Fine-Tuning for Open Intent Clustering through Dependency Parsing

POSTECH（浦项科技大学）

We present our work on Track 2 in the Dialog System Technology Challenges 11 (DSTC11). DSTC11-Track2 aims to provide a benchmark for zero-shot, cross-domain, intent-set induction. In the absence of in-domain training dataset, robust utterance representation that can be used across domains is necessary to induce users' intentions. To achieve this, we leveraged a multi-domain dialogue dataset to fine-tune the language model and proposed extracting Verb-Object pairs to remove the artifacts of unnecessary information. Furthermore, we devised the method that generates each cluster's name for the ex…

2023-03-17cs.AIcs.CL

Globally Normalized Reader

Baidu Silicon Valley Artificial Intelligence Lab（百度硅谷人工智能实验室）

Rapid progress has been made towards question answering (QA) systems that can extract answers from text. Existing neural approaches make use of expensive bi-directional attention mechanisms or score all possible answer spans, limiting scalability. We propose instead to cast extractive QA as an iterative search problem: select the answer's sentence, start word, and end word. This representation reduces the space of each search step and allows computation to be conditionally allocated to promising search paths. We show that globally normalizing the decision process and back-propagating through b…

2017-09-08cs.CL

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

AWS AI Labs（AWS人工智能实验室）

BERT model has been successfully applied to open-domain QA tasks. However, previous work trains BERT by viewing passages corresponding to the same question as independent training instances, which may cause incomparable scores for answers from different passages. To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages. In addition, we find that splitting articles into passages with the length of 100 words by sliding window improve…

2019-10-02cs.AIcs.CL

Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification

We introduce globally normalized convolutional neural networks for joint entity classification and relation extraction. In particular, we propose a way to utilize a linear-chain conditional random field output layer for predicting entity types and relations between entities at the same time. Our experiments show that global normalization outperforms a locally normalized softmax layer on a benchmark dataset.

2018-08-07cs.CL

An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search

Carnegie Mellon University（卡内基梅隆大学）；Deepmind（深度思考）；University of California, San Diego（加利福尼亚大学圣迭戈分校）

Globally normalized neural sequence models are considered superior to their locally normalized equivalents because they may ameliorate the effects of label bias. However, when considering high-capacity neural parametrizations that condition on the whole input sequence, both model classes are theoretically equivalent in terms of the distributions they are capable of representing. Thus, the practical advantage of global normalization in the context of modern neural methods remains unclear. In this paper, we attempt to shed light on this problem through an empirical study. We extend an approach f…

2019-04-15cs.CLcs.LGstat.ML

BLANC：通过多视图簇间归一化点间互信息变化发现专利空白区域

BLANC: Discovering Patent White Space via Changes in Normalized Pointwise Mutual Information Between Multi-View Clusters

AGC Inc.（AGC株式会社）

本研究提出BLANC三阶段流程，通过多视图神经主题建模、NPMI及ΔNPMI检测，在USPTO等语料库中成功恢复专利空白区域，其特异性优于单视图聚类与现有共现度量。

2026-08-28cs.CLcs.DLcs.IR

Global Normalization for Streaming Speech Recognition in a Modular Framework

Google Research（谷歌研究院）

We introduce the Globally Normalized Autoregressive Transducer (GNAT) for addressing the label bias problem in streaming speech recognition. Our solution admits a tractable exact computation of the denominator for the sequence-level normalization. Through theoretical and empirical results, we demonstrate that by switching to a globally normalized model, the word error rate gap between streaming and non-streaming speech-recognition models can be greatly reduced (by more than 50\% on the Librispeech dataset). This model is developed in a modular framework which encompasses all the common neural …

2022-05-26cs.AIcs.CLcs.LG

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

China University of Petroleum (East China)（中国石油大学（华东））；Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua Space Center（清华大学航天航空学院航天中心）；Linyi University（临沂大学）

Natural language processing (NLP) task has achieved excellent performance in many fields, including semantic understanding, automatic summarization, image recognition and so on. However, most of the neural network models for NLP extract the text in a fine-grained way, which is not conducive to grasp the meaning of the text from a global perspective. To alleviate the problem, the combination of the traditional statistical method and deep learning model as well as a novel model based on multi model nonlinear fusion are proposed in this paper. The model uses the Jaccard coefficient based on part …

2022-02-05cs.CL