基于上下文嵌入的卷积神经网络重复问句识别
Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification
摘要:
问题释义识别(QPI)是大型问答论坛的一项关键任务。QPI的目的是确定给定的一对问题在语义上是否相同。以前的方法已经取得了可喜的成果,但往往依赖于复杂的复发机制,是昂贵的和耗时的性质。在本文中,我们提出了一种新的架构相结合的双向Transformer编码器与卷积神经网络的QPI任务。我们使用两种不同的推理设置:Siamese和Matched Aggregation从所提出的架构中产生预测。实验结果表明,我们的模型在Quora Question Pairs数据集上达到了最先进的性能。我们凭经验证明,在模型架构中添加卷积层可以改善两种推理设置的结果。我们还研究了部分和完全微调的影响,并分析了计算能力和准确性之间的权衡。基于所获得的结果,我们得出结论,匹配聚合设置始终优于暹罗设置。我们的工作提供了什么样的架构组合和设置可能会产生更好的结果QPI任务的见解。
英文摘要:
Question Paraphrase Identification (QPI) is a critical task for large-scale Question-Answering forums. The purpose of QPI is to determine whether a given pair of questions are semantically identical or not. Previous approaches for this task have yielded promising results, but have often relied on complex recurrence mechanisms that are expensive and time-consuming in nature. In this paper, we propose a novel architecture combining a Bidirectional Transformer Encoder with Convolutional Neural Networks for the QPI task. We produce the predictions from the proposed architecture using two different inference setups: Siamese and Matched Aggregation. Experimental results demonstrate that our model achieves state-of-the-art performance on the Quora Question Pairs dataset. We empirically prove that the addition of convolution layers to the model architecture improves the results in both inference setups. We also investigate the impact of partial and complete fine-tuning and analyze the trade-off between computational power and accuracy in the process. Based on the obtained results, we conclude that the Matched-Aggregation setup consistently outperforms the Siamese setup. Our work provides insights into what architecture combinations and setups are likely to produce better results for the QPI task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。