arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.07743cs.CLcs.AIcs.LG
收藏

超越I.I.D.:基于知识库的问答的三层概括

Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases

The Ohio State University(俄亥俄州立大学);U.S. Army Research Laboratory(美国陆军研究实验室);Stanford University(斯坦福大学);University of California, Santa Barbara(加州大学圣巴巴拉分校)

Yu Gu, Sue Kase, Michelle Vanni, Brian Sadler, Percy Liang, Xifeng Yan, Yu Su

摘要:

现有的基于知识库的问答(KBQA)研究主要采用标准i.i.d假设,即,问题上的训练分布与测试分布相同。然而,i.i.d在大规模KB上可能既不合理地可实现也不理想,因为1)真实的用户分布难以捕获,以及2)从巨大的空间中随机采样训练样本将是高度数据低效的。相反,我们建议KBQA模型应该有三个级别的内置泛化:i.i.d,组合和zero-shot。为了促进具有更强泛化能力的KBQA模型的开发,我们构建并发布了一个包含64,331个问题的新的大规模高质量数据集GrailQA,并为所有三个泛化级别提供了评估设置。此外,我们提出了一种新的基于BERT的KBQA模型。我们的数据集和模型的结合使我们能够第一次彻底检查和展示预训练的上下文嵌入(如BERT)在KBQA泛化中的关键作用。

英文摘要:

Existing studies on question answering on knowledge bases (KBQA) mainly operate with the standard i.i.d assumption, i.e., training distribution over questions is the same as the test distribution. However, i.i.d may be neither reasonably achievable nor desirable on large-scale KBs because 1) true user distribution is hard to capture and 2) randomly sample training examples from the enormous space would be highly data-inefficient. Instead, we suggest that KBQA models should have three levels of built-in generalization: i.i.d, compositional, and zero-shot. To facilitate the development of KBQA models with stronger generalization, we construct and release a new large-scale, high-quality dataset with 64,331 questions, GrailQA, and provide evaluation settings for all three levels of generalization. In addition, we propose a novel BERT-based KBQA model. The combination of our dataset and model enables us to thoroughly examine and demonstrate, for the first time, the key role of pre-trained contextual embeddings like BERT in the generalization of KBQA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑