统一结构推理与语言模型预训练以实现复杂推理
Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoning
AI总结：
本文提出融合显式结构推理与语言预训练的统一框架，通过构建结构化查询并在PLM语义表示空间上逐步推理，提升模型跨结构复杂推理、少样本迁移及KG模态推理能力。
摘要：
近期具备基础推理能力的预训练语言模型（PLM）在下游复杂任务上表现出色。然而，重要的结构推理能力却很少得到研究；该能力涉及对文本中隐含的结构信息进行建模，并在这些信息之上执行显式逻辑推理以推出结论。本文提出一种统一学习框架，将显式结构推理与语言预训练相结合，使 PLM 具备结构推理能力。该框架首先识别上下文中的若干基本结构以构建结构化查询，并沿查询逐步推理，从而识别答案实体。文本语义与结构推理的融合通过以下方式实现：使用 PLM 学习到的上下文表示初始化结构的表示空间，并在该语义表示空间上执行逐步推理。在四个数据集上的实验结果表明，所提出模型在涉及多种结构的复杂推理任务上取得显著提升，并展现出在训练数据有限的下游任务中的可迁移性，以及对知识图谱（KG）模态复杂推理的有效性。
英文摘要：
Recent pre-trained language models (PLMs) equipped with foundation reasoning skills have shown remarkable performance on downstream complex tasks. However, the significant structure reasoning skill has been rarely studied, which involves modeling implicit structure information within the text and performing explicit logical reasoning over them to deduce the conclusion. This paper proposes a unified learning framework that combines explicit structure reasoning and language pre-training to endow PLMs with the structure reasoning skill. It first identifies several elementary structures within contexts to construct structured queries and performs step-by-step reasoning along the queries to identify the answer entity. The fusion of textual semantics and structure reasoning is achieved by using contextual representations learned by PLMs to initialize the representation space of structures, and performing stepwise reasoning on this semantic representation space. Experimental results on four datasets demonstrate that the proposed model achieves significant improvements in complex reasoning tasks involving diverse structures, and shows transferability to downstream tasks with limited training data and effectiveness for complex reasoning of KGs modality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
