The ability to reason over learned knowledge is an innate ability for humans and humans can easily master new reasoning rules with only a few demonstrations. While most existing studies on knowledge graph (KG) reasoning assume enough training examples, we study the challenging and practical problem of few-shot knowledge graph reasoning under the paradigm of meta-learning. We propose a new meta learning framework that effectively utilizes the task-specific meta information such as local graph neighbors and reasoning paths in KGs. Specifically, we design a meta-encoder that encodes the meta information into task-specific initialization parameters for different tasks. This allows our reasoning module to have diverse starting points when learning to reason over different relations, which is expected to better fit the target task. On two few-shot knowledge base completion benchmarks, we show that the augmented task-specific meta-encoder yields much better initial point than MAML and outperforms several few-shot learning baselines.
While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks. This paper presents a survey of these tasks, discusses the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation as revealed by these tasks, and reflects…
Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge is required during generation. Existing approaches that integrate commonsense knowledge into generative pre-trained language models simply transfer relational knowledge by post-training on individual knowledge triples while ignoring rich connections within the knowledge graph. We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. I…
Heidelberg University（海德堡大学）；Research Training Group AIPHES（AIPHES研究培训小组）；Institute for Computational Linguistics（计算语言学研究所）
Social Commonsense Reasoning requires understanding of text, knowledge about social events and their pragmatic implications, as well as commonsense reasoning skills. In this work we propose a novel multi-head knowledge attention model that encodes semi-structured commonsense inference rules and learns to incorporate them in a transformer-based reasoning cell. We assess the model's performance on two tasks that require different reasoning skills: Abductive Natural Language Inference and Counterfactual Invariance Prediction as a new task. We show that our proposed model improves performance over…
Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
Department of Computer Science, Yale University；Department of Molecular \& Cellular Biology, Harvard University；Allen Institute for AI；Department of Computer Science, Northwestern University
Multi-hop Knowledge Base Question Answering(KBQA) aims to find the answer entity in a knowledge base which is several hops from the topic entity mentioned in the question. Existing Retrieval-based approaches first generate instructions from the question and then use them to guide the multi-hop reasoning on the knowledge graph. As the instructions are fixed during the whole reasoning procedure and the knowledge graph is not considered in instruction generation, the model cannot revise its mistake once it predicts an intermediate entity incorrectly. To handle this, we propose KBIGER(Knowledge Ba…
Generative commonsense reasoning is the capability of a language model to generate a sentence with a given concept-set that is based on commonsense knowledge. However, generative language models still struggle to provide outputs, and the training set does not contain patterns that are sufficient for generative commonsense reasoning. In this paper, we propose a data-centric method that uses automatic knowledge augmentation to extend commonsense knowledge using a machine knowledge generator. This method can generate semi-golden sentences that improve the generative commonsense reasoning of a lan…
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models. To investigate this question, we develop generated knowledge prompting, which consists of generating knowledge from a language model, then providing the knowledge as additional input when answering a question. Our method does not require task-specific supervision for knowledge integration, or access to a structured knowledge base, yet it improves performance of large-scale, state-of-the-art models on four commonsense reasoning task…
KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Neusoft AI Magic Technology Research（东软AI魔法技术研究院）；Neusoft Institute of Intelligent Medical Research（东软智能医学研究院）
Microsoft Cognitive Services Research Group（微软认知服务研究组）
Commonsense reasoning (CSR) requires the model to be equipped with general world knowledge. While CSR is a language-agnostic process, most comprehensive knowledge sources are in few popular languages, especially English. Thus, it remains unclear how to effectively conduct multilingual commonsense reasoning (XCSR) for various languages. In this work, we propose to utilize English knowledge sources via a translate-retrieve-translate (TRT) strategy. For multilingual commonsense questions and choices, we collect related knowledge via translation and retrieval from the knowledge sources. The retrie…
The rapid development and application of natural language generation (NLG) techniques has revolutionized the field of automatic text production. However, these techniques are still limited in their ability to produce human-like text that is truly reasonable and informative. In this paper, we explore the importance of NLG being guided by knowledge, in order to convey human-like reasoning through language generation. We propose ten goals for intelligent NLG systems to pursue, and briefly review the achievement of NLG techniques guided by knowledge and reasoning. We also conclude by envisioning f…
OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases
Southeast University（东南大学）；Zhejiang University（浙江大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Tongji University（同济大学）；Monash University（莫纳什大学）；Soochow University（苏州大学）
Most of the existing work that focus on the identification of implicit knowledge in arguments generally represent implicit knowledge in the form of commonsense or factual knowledge. However, such knowledge is not sufficient to understand the implicit reasoning link between individual argumentative components (i.e., claim and premise). In this work, we focus on identifying the implicit knowledge in the form of argumentation knowledge which can help in understanding the reasoning link in arguments. Being inspired by the Argument from Consequences scheme, we propose a semi-structured template to …
Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, Korea（电气与计算机工程系，首尔国立大学，韩国首尔）；Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea（人工智能交叉学科项目，首尔国立大学，韩国首尔）
KnowRA: Knowledge Retrieval Augmented Method for Document-level Relation Extraction with Comprehensive Reasoning Abilities
Nanjing Normal University, School of Computer Science and Electronic Informatics（南京师范大学计算机科学与电子信息学院）；State Key Laboratory for Novel Software Technology at Nanjing University（南京大学新型软件技术国家重点实验室）；Nanjing University, School of Computer Science（南京大学计算机科学学院）
Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；School of Information Science and Engineering, Chongqing Jiaotong University（重庆交通大学信息科学与工程学院）；School of Information Engineering, Chang’an University（长安大学信息工程学院）
Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models
Institute for Artificial Intelligence, Tsinghua University (THUAI)（清华大学人工智能研究院）；Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心）；Department of Automation, Tsinghua University（清华大学自动化系）；TMLR Group, Hong Kong Baptist University（香港浸会大学TMLR组）
Pandora: Leveraging Code-driven Knowledge Transfer for Unified Structured Knowledge Reasoning
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（新一代人工智能技术及交叉应用重点实验室（东南大学））；China Mobile Research（中国移动研究院）；Monash University（莫纳什大学）
Huawei Poisson Laboratory（华为泊松实验室）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Hong Kong University of Science and Technology（香港科技大学）
Previous works show the great potential of pre-trained language models (PLMs) for storing a large amount of factual knowledge. However, to figure out whether PLMs can be reliable knowledge sources and used as alternative knowledge bases (KBs), we need to further explore some critical features of PLMs. Firstly, knowledge memorization and identification abilities: traditional KBs can store various types of entities and relationships; do PLMs have a high knowledge capacity to store different types of knowledge? Secondly, reasoning ability: a qualified knowledge source should not only provide a co…
CheckRLM: Effective Knowledge-Thought Coherence Checking in Retrieval-Augmented Reasoning
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Beijing Key Laboratory of Artificial Intelligence for Education（北京人工智能教育重点实验室）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Department of Computer Science and Technology, Institute for AI, Tsinghua University（清华大学人工智能研究院计算机科学与技术系）；Northeastern University（东北大学）
University of Southern California（南加州大学）；IIT Delhi（印度理工学院德里分校）；Tsinghua University（清华大学）；Adobe Research（Adobe研究院）
Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks. However, KG edge (fact) sparsity and noisy edge extraction/generation often hinder models from obtaining useful knowledge to reason over. To address these issues, we propose a new KG-augmented model: Hybrid Graph Network (HGN). Unlike prior methods, HGN learns to jointly contextualize extracted and generated knowledge by reasoning over both within a unified graph structure. Given the task input context and an extracted KG subgraph, HGN is trained to generate embeddings for t…
Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
The Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology, China（合肥工业大学大数据知识工程教育部重点实验室）；School of Computer Science and Information Engineering, Hefei University of Technology, China（合肥工业大学计算机与信息学院）；Nanjing University of Science and Technology, China（南京理工大学）；China Unicom Digital Technology Co., Ltd., Beijing, China（联通数字科技有限公司）；China Unicom Internet of Things Co., Ltd., Nanjing, China（联通物联网有限责任公司）；Shandong Inspur Science Research Institute, Jinan, China（山东浪潮科学研究院）；Griffith University, Australia（格里菲斯大学）
