虚拟知识库上的微分推理
Differentiable Reasoning over a Virtual Knowledge Base
摘要:
我们认为回答复杂的多跳问题,使用语料库作为一个虚拟的知识库(KB)的任务。特别是,我们描述了一个神经模块,DrKIT,遍历文本数据像一个KB,轻轻地以下的关系在语料库中提到的实体之间的路径。在每个步骤中,该模块使用稀疏矩阵TFIDF索引和最大内积搜索(MIPS)的提及的上下文表示的特殊索引的组合。该模块是可微分的,因此可以使用基于梯度的方法从自然语言输入开始端到端地训练整个系统。我们还描述了一个预训练方案的上下文表示编码器生成硬负的例子,使用现有的知识库。我们发现,DrKIT在MetaQA数据集中的3跳问题上将准确率提高了9个点,将基于文本和基于知识库的最新技术之间的差距缩小了70%。在HotpotQA上,DrKIT比基于BERT的重新排序方法在检索回答问题所需的相关段落方面提高了10%。DrKIT也非常高效,每秒处理的查询比现有的多跳系统多10- 100倍。
英文摘要:
We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. At each step the module uses a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions. This module is differentiable, so the full system can be trained end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the contextual representation encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. On HotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking approach to retrieving the relevant passages required to answer a question. DrKIT is also very efficient, processing 10-100x more queries per second than existing multi-hop systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。