arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.00102cs.CLcs.AI
收藏

比较、压缩和简化:用对齐因子分解增强神经结构用于自然语言推理

Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference

Nanyang Technological University(南洋理工大学);Institute for Infocomm Research, A*Star(新加坡科技研究局资讯通信研究院)

Yi Tay, Luu Anh Tuan, Siu Cheung Hui

摘要:

本文提出了一种新的用于自然语言推理(NLI)的深度学习架构。首先,我们介绍了一种新的架构,比对对进行比较,压缩,然后传播到上层增强表示学习。其次,我们采用因式分解层,将对齐向量高效且富有表现力地压缩为标量特征,然后将其用于增强基本单词表示。我们的方法的设计旨在概念上简单,紧凑,但功能强大。我们在三个流行的基准测试,SNLI,MultiNLI和SciTail上进行实验,在所有测试中都取得了有竞争力的性能。与现有的最先进的模型相比,我们的模型的轻量级参数化在参数大小上也减少了$\approx 3$倍,例如,ESIM和DIIN,同时保持竞争力。此外,视觉分析表明,我们传播的功能是高度可解释的。

英文摘要:

This paper presents a new deep learning architecture for Natural Language Inference (NLI). Firstly, we introduce a new architecture where alignment pairs are compared, compressed and then propagated to upper layers for enhanced representation learning. Secondly, we adopt factorization layers for efficient and expressive compression of alignment vectors into scalar features, which are then used to augment the base word representations. The design of our approach is aimed to be conceptually simple, compact and yet powerful. We conduct experiments on three popular benchmarks, SNLI, MultiNLI and SciTail, achieving competitive performance on all. A lightweight parameterization of our model also enjoys a $\approx 3$ times reduction in parameter size compared to the existing state-of-the-art models, e.g., ESIM and DIIN, while maintaining competitive performance. Additionally, visual analysis shows that our propagated features are highly interpretable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑