神经网络释义生成的粒度连续分解
Continuous Decomposition of Granularity for Neural Paraphrase Generation
摘要：
虽然Transformers在段落生成方面取得了巨大的成功，但它们将句子视为线性序列，并且经常忽略它们的层次信息。先前的工作已经表明，分解粒度级别（例如，单词、短语或句子）的输入标记的改进已经产生了实质性的改进，这表明了通过更细粒度的粒度建模来增强Transformers的可能性。在这项工作中，我们提出了一个连续的粒度分解神经释义生成（C-DNPG）。为了有效地将粒度纳入句子编码，C-DNPG引入了粒度感知注意力（GA-Attention）机制，该机制扩展了多头自注意，具有：1）粒度头，其通过神经估计每个输入标记的粒度级别来自动推断句子的层次结构;以及2）两种新的注意力掩码，即粒度共振和粒度范围，以有效地将粒度编码为注意力。在Quora问题对和Twitter URL等两个基准测试上的实验表明，C-DNPG的性能明显优于基线模型，并在许多指标方面取得了最先进的结果。定性分析表明，C-DNPG确实有效地捕获了细粒度级别的粒度。
英文摘要：
While Transformers have had significant success in paragraph generation, they treat sentences as linear sequences of tokens and often neglect their hierarchical information. Prior work has shown that decomposing the levels of granularity~(e.g., word, phrase, or sentence) for input tokens has produced substantial improvements, suggesting the possibility of enhancing Transformers via more fine-grained modeling of granularity. In this work, we propose a continuous decomposition of granularity for neural paraphrase generation (C-DNPG). In order to efficiently incorporate granularity into sentence encoding, C-DNPG introduces a granularity-aware attention (GA-Attention) mechanism which extends the multi-head self-attention with: 1) a granularity head that automatically infers the hierarchical structure of a sentence by neurally estimating the granularity level of each input token; and 2) two novel attention masks, namely, granularity resonance and granularity scope, to efficiently encode granularity into attention. Experiments on two benchmarks, including Quora question pairs and Twitter URLs have shown that C-DNPG outperforms baseline models by a remarkable margin and achieves state-of-the-art results in terms of many metrics. Qualitative analysis reveals that C-DNPG indeed captures fine-grained levels of granularity with effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
