This paper introduces context algebras and demonstrates their application to combining logical and vector-based representations of meaning. Other approaches to this problem attempt to reproduce aspects of logical semantics within new frameworks. The approach we present here is different: We show how logical semantics can be embedded within a vector space framework, and use this to combine distributional semantics, in which the meanings of words are represented as vectors, with logical semantics, in which the meaning of a sentence is represented as a logical form.
This work deals with the challenge of learning and reasoning over multi-hop question answering (QA). We propose a graph reasoning network based on the semantic structure of the sentences to learn cross paragraph reasoning paths and find the supporting facts and the answer jointly. The proposed graph is a heterogeneous document-level graph that contains nodes of type sentence (question, title, and other sentences), and semantic role labeling sub-graphs per sentence that contain arguments as nodes and predicates as edges. Incorporating the argument types, the argument phrases, and the semantics …
Institute of Discrete Mathematics and Geometry（离散数学与几何研究所）；Vienna University of Technology（维也纳工业大学）
Reasoning over streams of input data is an essential part of human intelligence. During the last decade {\em stream reasoning} has emerged as a research area within the AI-community with many potential applications. In fact, the increased availability of streaming data via services like Google and Facebook has raised the need for reasoning engines coping with data that changes at high rate. Recently, the rule-based formalism {\em LARS} for non-monotonic stream reasoning under the answer set semantics has been introduced. Syntactically, LARS programs are logic programs with negation incorporati…
Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Google Research（谷歌研究）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Politecnico di Milano（米兰理工学院）
Singapore Management University（新加坡管理大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Tencent AI Lab（腾讯人工智能实验室）；Microsoft（微软）
Commonsense reasoning is fundamental to natural language understanding. While traditional methods rely heavily on human-crafted features and knowledge bases, we explore learning commonsense knowledge from a large amount of raw text via unsupervised learning. We propose two neural network models based on the Deep Structured Semantic Models (DSSM) framework to tackle two classic commonsense reasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation (PDP). Evaluation shows that the proposed models effectively capture contextual information in the sentence and co-reference inform…
Knowledge retrieval and reasoning are two key stages in multi-hop question answering (QA) at web scale. Existing approaches suffer from low confidence when retrieving evidence facts to fill the knowledge gap and lack transparent reasoning process. In this paper, we propose a new framework to exploit more valid facts while obtaining explainability for multi-hop QA by dynamically constructing a semantic graph and reasoning over it. We employ Abstract Meaning Representation (AMR) as semantic graph representation. Our framework contains three new ideas: (a) {\tt AMR-SG}, an AMR-based Semantic Grap…
Beijing Normal University（北京师范大学）；Renmin University of China（中国人民大学）
Analogical reasoning is effective in capturing linguistic regularities. This paper proposes an analogical reasoning task on Chinese. After delving into Chinese lexical knowledge, we sketch 68 implicit morphological relations and 28 explicit semantic relations. A big and balanced dataset CA8 is then built for this task, including 17813 questions. Furthermore, we systematically explore the influences of vector representations, context features, and corpora on analogical reasoning. With the experiments, CA8 is proved to be a reliable benchmark for evaluating Chinese word embeddings.
Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph
Wangxuan Institute of Computer Technology, Peking University（王轩计算机技术研究所，北京大学）；Institute of Artificial Intelligence and Future Networks, Beijing Normal University（人工智能与未来网络研究院，北京师范大学）
Due to the lack of insufficient data, existing multi-hop open domain question answering systems require to effectively find out relevant supporting facts according to each question. To alleviate the challenges of semantic factual sentences retrieval and multi-hop context expansion, we present a semantic sentence composition reasoning approach for a multi-hop question answering task, which consists of two key modules: a multi-stage semantic matching module (MSSM) and a factual sentence composition module (FSC). With the combination of factual sentences and multi-stage semantic retrieval, our ap…
SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning
School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Merchants Union Consumer Finance Company Limited（招商局联融消费金融股份有限公司）
Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Models
Centro de Investigación en Ciencias, Universidad Autónoma del Estado de Morelos（莫雷洛斯州自治大学科学研究中心）
Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）
Probabilistic context free grammars (PCFG) have been the core of the probabilistic reasoning based parsers for several years especially in the context of the NLP. Multi entity bayesian networks (MEBN) a First Order Logic probabilistic reasoning methodology is widely adopted and used method for uncertainty reasoning. Further upper ontology like Probabilistic Ontology Web Language (PR-OWL) built using MEBN takes care of probabilistic ontologies which model and capture the uncertainties inherent in the domain's semantic information. The paper attempts to establish a link between probabilistic rea…
LLM-Guided Semantic Relational Reasoning for Multimodal Intent Recognition
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；School of Information Science and Engineering, Hebei University of Science and Technology（河北科技大学信息科学与工程学院）
Solving math word problems is a challenging task that requires accurate natural language understanding to bridge natural language texts and math expressions. Motivated by the intuition about how human generates the equations given the problem texts, this paper presents a neural approach to automatically solve math word problems by operating symbols according to their semantic meanings in texts. This paper views the process of generating equation as a bridge between the semantic world and the symbolic world, where the proposed neural math solver is based on an encoder-decoder framework. In the …
University of Southern California（南加州大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Allen Institute for AI（艾伦人工智能研究所）；Amazon（亚马逊）；University of Pennsylvania（宾夕法尼亚大学）
Understanding how events are semantically related to each other is the essence of reading comprehension. Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. While these tasks partially evaluate machines' ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning. For example, to understand causality between events, we need to infer motivation or purpose; to establish event hierarchy, we need to understand the composition of events. …
Singapore Management University（新加坡管理大学）；A*STAR, Institute for Infocomm Research（新加坡科技研究局资讯通信研究院）；Salesforce Research Asia（赛富时亚洲研究院）
Compared to traditional visual question answering, video-grounded dialogues require additional reasoning over dialogue context to answer questions in a multi-turn setting. Previous approaches to video-grounded dialogues mostly use dialogue context as a simple text input without modelling the inherent information flows at the turn level. In this paper, we propose a novel framework of Reasoning Paths in Dialogue Context (PDC). PDC model discovers information flows among dialogue turns through a semantic graph constructed based on lexical components in each question and answer. PDC model then lea…
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）
Natural language rationales could provide intuitive, higher-level explanations that are easily understandable by humans, complementing the more broadly studied lower-level explanations based on gradients or attention weights. We present the first study focused on generating natural language rationales across several complex visual reasoning tasks: visual commonsense reasoning, visual-textual entailment, and visual question answering. The key challenge of accurate rationalization is comprehensive image understanding at all levels: not just their explicit content at the pixel level, but their co…
Tencent Jarvis Lab（腾讯贾维斯实验室）；University of Oxford（牛津大学）；Tencent Platform and Content Group（腾讯平台与内容事业群）；Tencent Healthcare (Shenzhen) Co., LTD.（腾讯医疗健康（深圳）有限公司）
Knowledge Graph (KG) alignment aims at finding equivalent entities and relations (i.e., mappings) between two KGs. The existing approaches utilize either reasoning-based or semantic embedding-based techniques, but few studies explore their combination. In this demonstration, we present PRASEMap, an unsupervised KG alignment system that iteratively computes the Mappings with both Probabilistic Reasoning (PR) And Semantic Embedding (SE) techniques. PRASEMap can support various embedding-based KG alignment approaches as the SE module, and enables easy human computer interaction that additionally …
Sun Yat-sen University（中山大学）；Peking University（北京大学）；Microsoft Research（微软研究院）
Fact checking is a challenging task because verifying the truthfulness of a claim requires reasoning about multiple retrievable evidence. In this work, we present a method suitable for reasoning about the semantic-level structure of evidence. Unlike most previous works, which typically represent evidence sentences with either string concatenation or fusing the features of isolated evidence sentences, our approach operates on rich semantic structures of evidence obtained by semantic role labeling. We propose two mechanisms to exploit the structure of evidence while leveraging the advances of pr…
Sony Computer Science Laboratories Inc.（索尼计算机科学实验室公司）；University of Bremen（不来梅大学）
We present a system for generating and understanding of dynamic and static spatial relations in robotic interaction setups. Robots describe an environment of moving blocks using English phrases that include spatial relations such as "across" and "in front of". We evaluate the system in robot-robot interactions and show that the system can robustly deal with visual perception errors, language omissions and ungrammatical utterances.
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Pennsylvania（宾夕法尼亚大学）；Allen Institute for Artificial Intelligence (AI2)（艾伦人工智能研究所（AI2））
We propose a novel method for exploiting the semantic structure of text to answer multiple-choice questions. The approach is especially suitable for domains that require reasoning over a diverse set of linguistic constructs but have limited training data. To address these challenges, we present the first system, to the best of our knowledge, that reasons over a wide range of semantic abstractions of the text, which are derived using off-the-shelf, general-purpose, pre-trained natural language modules such as semantic role labelers, coreference resolvers, and dependency parsers. Representing mu…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Unisound Information Technology Co., Ltd（北京云知声信息技术有限公司）
Background Knowledge graphs (KGs), especially medical knowledge graphs, are often significantly incomplete, so it necessitating a demand for medical knowledge graph completion (MedKGC). MedKGC can find new facts based on the exited knowledge in the KGs. The path-based knowledge reasoning algorithm is one of the most important approaches to this task. This type of method has received great attention in recent years because of its high performance and interpretability. In fact, traditional methods such as path ranking algorithm (PRA) take the paths between an entity pair as atomic features. Howe…
Tencent Jarvis Lab（腾讯天衍实验室）；University of Oxford（牛津大学）；Tencent（腾讯）；Zhejiang University（浙江大学）
Knowledge Graph (KG) alignment is to discover the mappings (i.e., equivalent entities, relations, and others) between two KGs. The existing methods can be divided into the embedding-based models, and the conventional reasoning and lexical matching based systems. The former compute the similarity of entities via their cross-KG embeddings, but they usually rely on an ideal supervised learning setting for good performance and lack appropriate reasoning to avoid logically wrong mappings; while the latter address the reasoning issue but are poor at utilizing the KG graph structures and the entity c…
Georgia Institute of Technology（佐治亚理工学院）；SRI International（SRI国际）
This paper presents a novel approach for the Vision-and-Language Navigation (VLN) task in continuous 3D environments, which requires an autonomous agent to follow natural language instructions in unseen environments. Existing end-to-end learning-based VLN methods struggle at this task as they focus mostly on utilizing raw visual observations and lack the semantic spatio-temporal reasoning capabilities which is crucial in generalizing to new environments. In this regard, we present a hybrid transformer-recurrence model which focuses on combining classical semantic mapping techniques with a lear…
An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLM
The Graduate Center, CUNY（CUNY研究生中心）；Brooklyn College, CUNY（CUNY布鲁克林学院）；Borough of Manhattan Community College, CUNY（CUNY曼哈顿社区学院）；The City College of New York, CUNY（CUNY纽约城市学院）
