一种知识注入的课程预训练框架用于问答任务
A Knowledge-Injected Curriculum Pretraining Framework for Question Answering
AI总结：
提出一种通用的知识注入课程预训练框架（KICP），通过知识注入、知识适应和课程推理三个模块，实现全面知识图谱学习与利用，在KBQA任务上取得更高性能。
摘要：
基于知识的问答（KBQA）是自然语言处理研究中的一项关键任务，也是访问网络数据和知识的一种途径，它要求利用知识图谱（KGs）进行推理。在文献中，KBQA的一个有前景的解决方案是通过生成以知识图谱为中心的预训练语料，将预训练语言模型（LM）与知识图谱相结合，该方法已显示出其优越性。然而，这些方法往往依赖于特定的技术和资源才能工作，这些技术和资源可能并非总是可用，从而限制了其应用。此外，现有方法更侧重于利用知识图谱提升语言理解能力，而忽视了更为重要的人类式复杂推理。为此，在本文中，我们提出了一种通用的知识注入的课程预训练框架（KICP），以实现对KBQA任务的全面知识图谱学习和利用，该框架由知识注入（KI）、知识适应（KA）和课程推理（CR）三个模块组成。具体而言，KI模块首先通过生成以知识图谱为中心的预训练语料将知识注入语言模型，并将该过程概括为三个关键步骤，这些步骤可与不同的实现方式配合使用，以实现灵活应用。接下来，KA模块利用配备适配器的语言模型从生成的语料中学习知识，同时保持其原有的自然语言理解能力，以减少生成语料与自然语料之间差异带来的负面影响。最后，为使语言模型具备复杂推理能力，CR模块遵循人类推理模式，构建三个推理难度递增的语料，并以课程学习的方式从易到难对语言模型进行训练。我们提供了该通用框架的一种实现，并在四个真实数据集上评估了所提出的KICP。结果表明，我们的框架能够取得更高的性能。
英文摘要：
Knowledge-based question answering (KBQA) is a key task in NLP research, and also an approach to access the web data and knowledge, which requires exploiting knowledge graphs (KGs) for reasoning. In the literature, one promising solution for KBQA is to incorporate the pretrained language model (LM) with KGs by generating KG-centered pretraining corpus, which has shown its superiority. However, these methods often depend on specific techniques and resources to work, which may not always be available and restrict its application. Moreover, existing methods focus more on improving language understanding with KGs, while neglect the more important human-like complex reasoning. To this end, in this paper, we propose a general Knowledge-Injected Curriculum Pretraining framework (KICP) to achieve comprehensive KG learning and exploitation for KBQA tasks, which is composed of knowledge injection (KI), knowledge adaptation (KA) and curriculum reasoning (CR). Specifically, the KI module first injects knowledge into the LM by generating KG-centered pretraining corpus, and generalizes the process into three key steps that could work with different implementations for flexible application. Next, the KA module learns knowledge from the generated corpus with LM equipped with an adapter as well as keeps its original natural language understanding ability to reduce the negative impacts of the difference between the generated and natural corpus. Last, to enable the LM with complex reasoning, the CR module follows human reasoning patterns to construct three corpora with increasing difficulties of reasoning, and further trains the LM from easy to hard in a curriculum manner. We provide an implementation of the general framework, and evaluate the proposed KICP on four real-word datasets. The results demonstrate that our framework can achieve higher performances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
