arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1711.07656cs.CLcs.AIcs.IR
收藏

用于问题-答案对排序的跨时间递归网络

Cross Temporal Recurrent Networks for Ranking Question Answer Pairs

Institute for Infocomm Research(资讯通信研究院);Nanyang Technological University(南洋理工大学)

Yi Tay, Luu Anh Tuan, Siu Cheung Hui

摘要:

时间门在现代基于递归的神经编码器中发挥着重要作用,可以随着时间的推移对递归组合操作进行细粒度控制。在诸如长短期记忆(LSTM)之类的递归模型中,时间门控制着随着时间的推移保留或丢弃的信息量,不仅在影响学习的表示方面发挥着重要作用,而且还可以防止梯度消失。本文探讨了学习时序门序列对(问题和答案)的想法,共同影响学习的表示成对的方式。在我们的方法中,时间门通过1D卷积层学习,然后交叉应用于联合学习的问题和答案。从经验上讲,我们表明,这种概念上简单的共享时间门可以导致竞争力的性能在多个基准。直观地说,我们的网络所实现的可以被解释为学习问题和答案对的表示,这些问题和答案对知道彼此在记住或忘记什么,即,成对时间门控通过大量的实验,我们表明,我们提出的模型在两个基于社区的QA数据集上实现了最先进的性能,在一个基于事实的QA数据集上实现了具有竞争力的性能。

英文摘要:

Temporal gates play a significant role in modern recurrent-based neural encoders, enabling fine-grained control over recursive compositional operations over time. In recurrent models such as the long short-term memory (LSTM), temporal gates control the amount of information retained or discarded over time, not only playing an important role in influencing the learned representations but also serving as a protection against vanishing gradients. This paper explores the idea of learning temporal gates for sequence pairs (question and answer), jointly influencing the learned representations in a pairwise manner. In our approach, temporal gates are learned via 1D convolutional layers and then subsequently cross applied across question and answer for joint learning. Empirically, we show that this conceptually simple sharing of temporal gates can lead to competitive performance across multiple benchmarks. Intuitively, what our network achieves can be interpreted as learning representations of question and answer pairs that are aware of what each other is remembering or forgetting, i.e., pairwise temporal gating. Via extensive experiments, we show that our proposed model achieves state-of-the-art performance on two community-based QA datasets and competitive performance on one factoid-based QA dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑