How perception and reasoning arise from neuronal network activity is poorly understood. This is reflected in the fundamental limitations of connectionist artificial intelligence, typified by deep neural networks trained via gradient-based optimization. Despite success on many tasks, such networks remain unexplainable black boxes incapable of symbolic reasoning and concept generalization. Here we show that a simple set of biologically consistent organizing principles confer these capabilities to neuronal networks. To demonstrate, we implement these principles in a novel machine learning algorithm, based on concept construction instead of optimization, to design deep neural networks that reason with explainable neuron activity. On a range of tasks including NP-hard problems, their reasoning capabilities grant additional cognitive functions, like deliberating through self-analysis, tolerating adversarial attacks, and learning transferable rules from simple examples to solve problems of unencountered complexity. The networks also naturally display properties of biological nervous systems inherently absent in current deep neural networks, including sparsity, modularity, and both distributed and localized firing patterns. Because they do not sacrifice performance, compactness, or training time on standard learning tasks, these networks provide a new black-box-free approach to artificial intelligence. They likewise serve as a quantitative framework to understand the emergence of cognition from neuronal networks.
Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Tech. R&D（陕西省卫星与地面网络技术研发重点实验室）；National Engineering Lab for Big Data Analytics（国家大数据分析工程实验室）
Logical reasoning task involves diverse types of complex reasoning over text, based on the form of multiple-choice question answering. Given the context, question and a set of options as the input, previous methods achieve superior performances on the full-data setting. However, the current benchmark dataset has the ideal assumption that the reasoning type distribution on the train split is close to the test split, which is inconsistent with many real application scenarios. To address it, there remain two problems to be studied: (1) How is the zero-shot capability of the models (train on seen …
Perception and reasoning are basic human abilities that are seamlessly connected as part of human intelligence. However, in current machine learning systems, the perception and reasoning modules are incompatible. Tasks requiring joint perception and reasoning ability are difficult to accomplish autonomously and still demand human intervention. Inspired by the way language experts decoded Mayan scripts by joining two abilities in an abductive manner, this paper proposes the abductive learning framework. The framework learns perception and reasoning simultaneously with the help of a trial-and-er…
Microsoft Applied Sciences Group (ASG)（微软应用科学集团）；Microsoft Research AI（微软人工智能研究院）
Visual reasoning tasks such as visual question answering (VQA) require an interplay of visual perception with reasoning about the question semantics grounded in perception. However, recent advances in this area are still primarily driven by perception improvements (e.g. scene graph generation) rather than reasoning. Neuro-symbolic models such as Neural Module Networks bring the benefits of compositional reasoning to VQA, but they are still entangled with visual representation learning, and thus neural reasoning is hard to improve and assess on its own. To address this, we propose (1) a framewo…
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China（中国科学技术大学）；State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；iFLYTEK AI Research（iFLYTEK人工智能研究院）；Hong Kong University of Science and Technology（香港科学与技术大学）
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Zhejiang University（浙江大学）
Position: Reasoning After Perception Means Reasoning Without Vision
Tsinghua University（清华大学）；Peking University（北京大学）；National University of Singapore（新加坡国立大学）；Zhongguancun Academy（中关村学院）；Nanjing University（南京大学）；Nankai University（南开大学）
Omni-Perception Policy Optimization for Multimodal Emotion Reasoning
University of Science and Technology of China（中国科学技术大学）；SenseTime Research（商汤科技研究院）；National University of Singapore（新加坡国立大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents
HyperAI Team, Xiaomi Corporation（HyperAI团队，小米公司）；Zhongnan University of Economics and Law（中南财经政法大学）；Jilin University（吉林大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
Saarland University（萨尔兰大学）；Deutsches Forschungszentrum für Künstliche Intelligenz（德国人工智能研究中心）；Télécom Paris（巴黎电信学院）；Max Planck Institute for Informatics（马克斯·普朗克 informatics 研究所）
Institute of Computer Science, Foundation for Research and Technology, Hellas（希腊研究与技术基金会计算机科学研究所）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Object perception is a fundamental sub-field of Computer Vision, covering a multitude of individual areas and having contributed high-impact results. While Machine Learning has been traditionally applied to address related problems, recent works also seek ways to integrate knowledge engineering in order to expand the level of intelligence of the visual interpretation of objects, their properties and their relations with their environment. In this paper, we attempt a systematic investigation of how knowledge-based methods contribute to diverse object perception tasks. We review the latest achie…
VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding
Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Shenzhen University（深圳大学）；University of Science and Technology of China（中国科学技术大学）
From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning
Zhejiang University（浙江大学）；Tianjin University（天津大学）；Qingdao University（青岛大学）；Shanghai Jiao Tong University（上海交通大学）；National University of Singapore（新加坡国立大学）
Georgia Institute of Technology（佐治亚理工学院）；Microsoft Research（微软研究院）
Existing VQA datasets contain questions with varying levels of complexity. While the majority of questions in these datasets require perception for recognizing existence, properties, and spatial relationships of entities, a significant portion of questions pose challenges that correspond to reasoning tasks - tasks that can only be answered through a synthesis of perception and knowledge about the world, logic and / or reasoning. Analyzing performance across this distinction allows us to notice when existing VQA models have consistency issues; they answer the reasoning questions correctly but f…
XEmoGPT: An Explainable Multimodal Emotion Recognition Framework with Cue-Level Perception and Reasoning
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；th Research Institute, China Electronics Technology Group Corporation（中国电子科技集团第五十四研究所）
ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统认知与决策智能重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；ETH Zurich（苏黎世联邦理工学院）；ETH AI Center（苏黎世联邦理工学院人工智能中心）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；Yinwang Intelligent Technology Co., Ltd.（云网智能技术有限公司）；Peking University and Beijing International Center for Mathematical Research（北京大学和北京国际数学研究中心）
