首页/论文/1806.04466

arXiv 1806.044662018-06-12cs.CL

Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model

Soochow University（苏州大学）

Shaohui Kuang, Deyi Xiong

备注：: Accepted by COLING2018.11 pages,4 figures. arXiv admin note: text overlap with arXiv:1711.11221

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Neural machine translation (NMT) systems are usually trained on a large amount of bilingual sentence pairs and translate one sentence at a time, ignoring inter-sentence information. This may make the translation of a sentence ambiguous or even inconsistent with the translations of neighboring sentences. In order to handle this issue, we propose an inter-sentence gate model that uses the same encoder to encode two adjacent sentences and controls the amount of information flowing from the preceding sentence to the translation of the current sentence with an inter-sentence gate. In this way, our proposed model can capture the connection between sentences and fuse recency from neighboring sentences into neural machine translation. On several NIST Chinese-English translation tasks, our experiments demonstrate that the proposed inter-sentence gate model achieves substantial improvements over the baseline.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Modeling Context With Linear Attention for Scalable Document-Level Translation

MIT（麻省理工学院）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；AWS AI（亚马逊云科技人工智能）；University of Washington（华盛顿大学）

Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transformers, are difficult to scale to long documents as their attention layers have quadratic complexity in the sequence length. Recent efforts on efficient attention improve scalability, but their effect on document translation remains unexplored. In this work, we investigate the efficacy of a recent linear attention model by Peng et al. (2021) on document translation and augment it with a sentential gate to promote a re…

2022-10-16cs.CL

Understanding Points of Correspondence between Sentences for Abstractive Summarization

University of Central Florida（中佛罗里达大学）；Adobe Research（奥多比研究院）

Fusing sentences containing disparate content is a remarkable human ability that helps create informative and succinct summaries. Such a simple task for humans has remained challenging for modern abstractive summarizers, substantially restricting their applicability in real-world scenarios. In this paper, we present an investigation into fusing sentences drawn from a document by introducing the notion of points of correspondence, which are cohesive devices that tie any two sentences together into a coherent text. The types of points of correspondence are delineated by text cohesion theory, cov…

2020-06-10cs.CL

通过多知识融合改进基于 LLM 的文档级机器翻译

Improving LLM-based Document-level Machine Translation with Multi-Knowledge Fusion

School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Huawei Translation Services Center（华为翻译服务中心）

本文提出融合文档摘要与实体翻译的多知识策略，用于改进基于 LLM 的文档级机器翻译，并在八项任务及三个模型上验证其 COMET 提升。

2025-03-15cs.CL

LabelFusion-TS：融合大语言模型、Transformer编码器与金融时间序列以进行货币政策立场分类

LabelFusion-TS: Fusing Large Language Models, Transformer Encoders, and Financial Time Series for Monetary-Policy Stance Classification

Georg-August-Universität Göttingen（哥廷根大学）；Hochschule Bielefeld (HSBI) - University of Applied Sciences and Arts（比勒费尔德应用科学与艺术大学）

本研究提出LabelFusion-TS系统，融合LLM、RoBERTa编码器与时间序列Transformer，利用金融时间序列辅助美联储沟通语句的货币政策立场分类，在2015-2022年数据上加权F1达70.2%，仅需少量人工标注即可超越零样本LLM。

2026-08-13cs.CL

Multi-layer Representation Fusion for Neural Machine Translation

Northeastern University（东北大学）；YingKou Institute of Technology（营口理工学院）

Neural machine translation systems require a number of stacked layers for deep models. But the prediction depends on the sentence representation of the top-most layer with no access to low-level representations. This makes it more difficult to train the model and poses a risk of information loss to prediction. In this paper, we propose a multi-layer representation fusion (MLRF) approach to fusing stacked layers. In particular, we design three fusion functions to learn a better representation from the stack. Experimental results show that our approach yields improvements of 0.92 and 0.56 BLEU p…

2020-02-16cs.CL

Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models

Universität des Saarlandes（萨尔大学）

Although masked language models are highly performant and widely adopted by NLP practitioners, they can not be easily used for autoregressive language modelling (next word prediction and sequence probability estimation). We present an LSTM-based autoregressive language model which uses prefix embeddings (from a pretrained masked language model) via fusion (e.g. concatenation) to obtain a richer context representation for language modelling. We find that fusion helps reliably in lowering the perplexity (16.74 $\rightarrow$ 15.80), which is even preserved after a transfer to a dataset from a dif…

2022-08-05cs.CLcs.LG

更多相关论文收起更多论文

FCDS：将成分句法和依存句法融合到文档级关系抽取中

FCDS: Fusing Constituency and Dependency Syntax into Document-Level Relation Extraction

University of Electronic Science and Technology of China（电子科技大学）

针对文档级关系抽取未能充分利用丰富句法信息的问题，提出 FCDS，融合成分句法与依存句法进行句子筛选、图增强和实体路径选择，并在多领域数据集上验证有效。

2024-03-04cs.AIcs.CL

Graph-to-Sequence Neural Machine Translation

Shanghai Jiao Tong University（上海交通大学）；National Institute of Information and Communications Technology (NICT)（日本信息通信研究机构）

Neural machine translation (NMT) usually works in a seq2seq learning way by viewing either source or target sentence as a linear sequence of words, which can be regarded as a special case of graph, taking words in the sequence as nodes and relationships between words as edges. In the light of the current NMT models more or less capture graph information among the sequence in a latent way, we present a graph-to-sequence model facilitating explicit graph information capturing. In detail, we propose a graph-based SAN-based NMT model called Graph-Transformer by capturing information of subgraphs o…

2020-09-16cs.CL

InSRL: A Multi-view Learning Framework Fusing Multiple Information Sources for Distantly-supervised Relation Extraction

University of Virginia（弗吉尼亚大学）；Tencent AI Lab（腾讯AI实验室）；Fudan University（复旦大学）

Distant supervision makes it possible to automatically label bags of sentences for relation extraction by leveraging knowledge bases, but suffers from the sparse and noisy bag issues. Additional information sources are urgently needed to supplement the training data and overcome these issues. In this paper, we introduce two widely-existing sources in knowledge bases, namely entity descriptions, and multi-grained entity types to enrich the distantly supervised data. We see information sources as multiple views and fusing them to construct an intact space with sufficient information. An end-to-e…

2020-12-17cs.CL

Abstractive Summarization Guided by Latent Hierarchical Document Structure

Institute for Language, Cognition and Computation（语言、认知与计算研究所）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）

Sequential abstractive neural summarizers often do not use the underlying structure in the input article or dependencies between the input sentences. This structure is essential to integrate and consolidate information from different parts of the text. To address this shortcoming, we propose a hierarchy-aware graph neural network (HierGNN) which captures such dependencies through three main steps: 1) learning a hierarchical document structure through a latent structure tree learned by a sparse matrix-tree computation; 2) propagating sentence information over this structure using a novel messag…

2022-11-17cs.CL

Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims

University of Chinese Academy of Sciences（中国科学院大学）；Key Lab of Data Engineering and Knowledge Engineering, Renmin University of China（中国人民大学数据工程与知识工程重点实验室）

False claims that have been previously fact-checked can still spread on social media. To mitigate their continual spread, detecting previously fact-checked claims is indispensable. Given a claim, existing works focus on providing evidence for detection by reranking candidate fact-checking articles (FC-articles) retrieved by BM25. However, these performances may be limited because they ignore the following characteristics of FC-articles: (1) claims are often quoted to describe the checked events, providing lexical information besides semantics; (2) sentence templates to introduce or debunk clai…

2021-12-20cs.CLcs.IR