The predominant approach to visual question answering (VQA) relies on encoding the image and question with a "black-box" neural encoder and decoding a single token as the answer like "yes" or "no". Despite this approach's strong quantitative results, it struggles to come up with intuitive, human-readable forms of justification for the prediction process. To address this insufficiency, we reformulate VQA as a full answer generation task, which requires the model to justify its predictions in natural language. We propose LRTA [Look, Read, Think, Answer], a transparent neural-symbolic reasoning framework for visual question answering that solves the problem step-by-step like humans and provides human-readable form of justification at each step. Specifically, LRTA learns to first convert an image into a scene graph and parse a question into multiple reasoning instructions. It then executes the reasoning instructions one at a time by traversing the scene graph using a recurrent neural-symbolic execution module. Finally, it generates a full answer to the given question with natural language justifications. Our experiments on GQA dataset show that LRTA outperforms the state-of-the-art model by a large margin (43.1% v.s. 28.0%) on the full answer generation task. We also create a perturbed GQA test set by removing linguistic cues (attributes and relations) in the questions for analyzing whether a model is having a smart guess with superficial data correlations. We show that LRTA makes a step towards truly understanding the question while the state-of-the-art model tends to learn superficial correlations from the training data.
Discourse vs emissions: Analysis of corporate narratives, symbolic practices, and mimicry through LLMs
QUANT AI Lab（QUANT AI 实验室）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Centre d’Économie de la Sorbonne (CES), Maison des Sciences Économiques (MSE), Université Paris 1 Panthéon-Sorbonne（索邦大学经济研究中心（CES）、经济科学之家（MSE））；Institut Louis Bachelier, Palais Brongniart, 28 Place de la Bourse, 75002 Paris, France（路易·巴希尔研究所）；EDC Paris Business School（巴黎商学院）
Plymouth University（普利茅斯大学）；Bernstein Center for Computational Neuroscience Berlin（柏林伯恩斯坦计算神经科学中心）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Inria Sophia-Antipolis Méditerranée（法国国家信息与自动化研究所索菲亚-安蒂波利斯地中海研究中心）
Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as to transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simul…
Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies
Department of Computer Science and Engineering, Birbhum Institute of Engineering and Technology（计算机科学与工程系，比罗尔理工学院）；College of Computer Science and Artificial Intelligence, Fudan University（计算机科学与人工智能学院，复旦大学）；Faculty of Computer Science and Information Technology, Universiti Malaya（计算机科学与信息技术学院，马来亚大学）
MolEvolve: LLM-Guided Evolutionary Search for Interpretable Molecular Optimization
Hong Kong University of Science（香港科技大学）；Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issues
Faculty of Information Technology, Monash University, Australia（墨尔本大学信息技术学院，澳大利亚）；School of Information Technology, Monash University Malaysia（墨尔本大学马来西亚分校信息学院）；School of Business, Monash University Malaysia（墨尔本大学马来西亚分校商学院）；Faculty of Law, Monash University, Australia（墨尔本大学法学院，澳大利亚）
We study the interpretability issue of task-oriented dialogue systems in this paper. Previously, most neural-based task-oriented dialogue systems employ an implicit reasoning strategy that makes the model predictions uninterpretable to humans. To obtain a transparent reasoning process, we introduce neuro-symbolic to perform explicit reasoning that justifies model decisions by reasoning chains. Since deriving reasoning chains requires multi-hop reasoning for task-oriented dialogues, existing neuro-symbolic approaches would induce error propagation due to the one-phase design. To overcome this, …
Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；Institute of Logic and Computation, TU Wien（维也纳技术大学逻辑与计算研究所）
A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
University of Information Technology (UIT)（信息科技大学）；Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学）；Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市分校）；University of Economics Ho Chi Minh City (UEH)（胡志明市经济大学）
Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Models
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）
University of Southern California（南加州大学）；Shanghai Jiao Tong University（上海交通大学）
Commonsense reasoning aims to empower machines with the human ability to make presumptions about ordinary situations in our daily life. In this paper, we propose a textual inference framework for answering commonsense questions, which effectively utilizes external, structured commonsense knowledge graphs to perform explainable inferences. The framework first grounds a question-answer pair from the semantic space to the knowledge-based symbolic space as a schema graph, a related sub-graph of external knowledge graphs. It represents schema graphs with a novel knowledge-aware graph network module…
