arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.09983cs.CL
收藏

多跳检索器的弱监督预训练

Weakly Supervised Pre-Training for Multi-Hop Retriever

KAIST(韩国科学技术院);NAVER AI Lab(NAVER人工智能实验室);NAVER Clova

Yeon Seonwoo, Sang-Woo Lee, Ji-Hoon Kim, Jung-Woo Ha, Alice Oh

摘要:

在多跳QA中,回答复杂的问题需要迭代的文档检索来找到问题的缺失实体。这个过程的主要步骤是子问题检测,子问题的文档检索,并生成一个新的查询,最终的文档检索。然而,构建包含复杂问题和子问题及其相应文档的数据集需要昂贵的人工注释。为了解决这个问题,我们提出了一种新的方法,弱监督多跳检索器预训练,而无需人工努力。我们的方法包括1)用于生成复杂问题的向量表示的预训练任务,2)可扩展的数据生成方法,该方法生成问题和子问题的嵌套结构作为预训练的弱监督,以及3)基于密集编码器的预训练模型结构。我们进行实验,比较我们的预训练检索器的性能与几个国家的最先进的模型端到端的多跳QA以及文档检索。实验结果表明,我们的预训练检索器是有效的,也是强大的有限的数据和计算资源。

英文摘要:

In multi-hop QA, answering complex questions entails iterative document retrieval for finding the missing entity of the question. The main steps of this process are sub-question detection, document retrieval for the sub-question, and generation of a new query for the final document retrieval. However, building a dataset that contains complex questions with sub-questions and their corresponding documents requires costly human annotation. To address the issue, we propose a new method for weakly supervised multi-hop retriever pre-training without human efforts. Our method includes 1) a pre-training task for generating vector representations of complex questions, 2) a scalable data generation method that produces the nested structure of question and sub-question as weak supervision for pre-training, and 3) a pre-training model structure based on dense encoders. We conduct experiments to compare the performance of our pre-trained retriever with several state-of-the-art models on end-to-end multi-hop QA as well as document retrieval. The experimental results show that our pre-trained retriever is effective and also robust on limited data and computational resources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑