学习构建特定任务的树结构
Learning to Compose Task-Specific Tree Structures
摘要:
多年来,递归神经网络(RvNN)已被证明适合将文本表示为固定长度的向量,并在多个自然语言处理任务上取得了良好的性能。然而,RvNN的主要缺点是它们需要结构化的输入,这使得数据准备和模型实现变得困难。在本文中,我们提出了Gumbel Tree-LSTM,这是一种新型的树结构长短期记忆体系结构,它可以学习如何有效地仅从纯文本数据组成特定于任务的树结构。我们的模型使用直通Gumbel-Softmax估计器来动态地确定候选节点中的父结点,并计算离散决策的梯度。我们评估所提出的模型对自然语言推理和情感分析,并表明,我们的模型优于或至少与以前的模型。我们还发现,我们的模型收敛速度明显快于其他模型。
英文摘要:
For years, recursive neural networks (RvNNs) have been shown to be suitable for representing text into fixed-length vectors and achieved good performance on several natural language processing tasks. However, the main drawback of RvNNs is that they require structured input, which makes data preparation and model implementation hard. In this paper, we propose Gumbel Tree-LSTM, a novel tree-structured long short-term memory architecture that learns how to compose task-specific tree structures only from plain text data efficiently. Our model uses Straight-Through Gumbel-Softmax estimator to decide the parent node among candidates dynamically and to calculate gradients of the discrete decision. We evaluate the proposed model on natural language inference and sentiment analysis, and show that our model outperforms or is at least comparable to previous models. We also find that our model converges significantly faster than other models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。