Existing research of video understanding still struggles to achieve in-depth comprehension and reasoning in complex videos, primarily due to the under-exploration of two key bottlenecks: fine-grained spatial-temporal perceptive understanding and cognitive-level video scene comprehension. This paper bridges the gap by presenting a novel solution. We first introduce a novel video Multimodal Large Language Model (MLLM), MotionEpic, which achieves fine-grained pixel-level spatial-temporal video grounding by integrating video spatial-temporal scene graph (STSG) representation. Building upon MotionEpic, we then develop a Video-of-Thought (VoT) reasoning framework. VoT inherits the Chain-of-Thought (CoT) core, breaking down a complex task into simpler and manageable sub-problems, and addressing them step-by-step from a low-level pixel perception to high-level cognitive interpretation. Extensive experiments across various complex video QA benchmarks demonstrate that our overall framework strikingly boosts existing state-of-the-art. To our knowledge, this is the first attempt at successfully implementing the CoT technique for achieving human-level video reasoning, where we show great potential in extending it to a wider range of video understanding scenarios. Project is open at https://haofei.vip/VoT
Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University（浙江可感知智能系统重点实验室，浙江大学）；Alibaba Group（阿里巴巴集团）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and DataSecurity（杭州高新技术区（滨江）区块链与数据安全研究院）
From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院）；Xiamen University（厦门大学）；The Hong Kong University of Science and Technology（香港理工大学）；Nanyang Technological University（南洋理工大学）
Metareasoning constraints couple narratives, affect and cognition
Applied Computational Psychiatry Lab（应用计算精神病学实验室）；Max Planck UCL Centre for Computational Psychiatry and Ageing Research（马克斯·普朗克UCL计算精神病学与衰老研究中心）；Queen Square Institute of Neurology and Mental Health（圣夸克广场神经病学与心理健康研究所）；Neuroscience Department（神经科学系）；Division of Psychiatry（精神病学系）
VisDoT : Enhancing Visual Reasoning through Human-Like Interpretation Grounding and Decomposition of Thought
Department of Computer Science and Artificial Intelligence, Dongguk University（东国大学计算机科学与人工智能系）；Department of Electronics and Electrical Engineering, Dongguk University（东国大学电子与电气工程系）
Emotional intelligence in large language models is fragmented across perception, cognition, and interaction
X-LANCE Lab（X-LANCE实验室）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；MoE Key Lab of Artificial Intelligence（人工智能MOE重点实验室）；Jiangsu Key Lab of Language Computing（江苏省语言计算重点实验室）；Beijing Key Laboratory of Applied Experimental Psychology（北京应用实验心理学重点实验室）；National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University（北京师范大学实验心理学教育国家级示范中心，心理学学院）
Independent Researcher（独立研究者）；Department of Computing Science, University of Alberta, Canada（阿尔伯塔大学计算机科学系）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Sun Yat-sen University（中山大学）；Shenzhen Loop Area Institute（深圳河套学院）；Beijing Zhongguancun Academy（北京中关村研究院）；Huawei Technologies Co., Ltd（华为技术有限公司）；Harbin Institute of Technology（哈尔滨工业大学）
University of Central Florida（中佛罗里达大学）；InfoTech Labs, Toyota Motor North America（丰田汽车北美公司InfoTech实验室）
The recent advancement in computational and communication systems has led to the introduction of high-performing neural networks and high-speed wireless vehicular communication networks. As a result, new technologies such as cooperative perception and cognition have emerged, addressing the inherent limitations of sensory devices by providing solutions for the detection of partially occluded targets and expanding the sensing range. However, designing a reliable cooperative cognition or perception system requires addressing the challenges caused by limited network resources and discrepancies bet…
Understanding, Protecting, and Augmenting Human Cognition with Generative AI: A Synthesis of the CHI 2025 Tools for Thought Workshop
Microsoft Research（微软研究院）；Harvard University（哈佛大学）；IBM Research（IBM研究院）；Carnegie Mellon University（卡内基梅隆大学）；University of Chicago（芝加哥大学）；Tableau Research（Tableau研究院）；University College London（伦敦大学学院）；Stanford University（斯坦福大学）
Cisco Systems（思科系统公司）；Icelandic Institute for Intelligent Machines（冰岛智能机器研究所）；Reykjavik University（雷克雅未克大学）；Temple University（天普大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Stockholm University（斯德哥尔摩大学）
A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of …
Brussels Free University（布鲁塞尔自由大学）；University of British Columbia（不列颠哥伦比亚大学）
Experiments in cognitive science and decision theory show that the ways in which people combine concepts and make decisions cannot be described by classical logic and probability theory. This has serious implications for applied disciplines such as information retrieval, artificial intelligence and robotics. Inspired by a mathematical formalism that generalizes quantum mechanics the authors have constructed a contextual framework for both concept representation and decision making, together with quantum models that are in strong alignment with experimental data. The results can be interpreted …
Center Leo Apostel for Interdisciplinary Studies（利奥·阿波斯特尔跨学科研究中心）；Brussels Free University（布鲁塞尔自由大学）
We identify the presence of typically quantum effects, namely 'superposition' and 'interference', in what happens when human concepts are combined, and provide a quantum model in complex Hilbert space that represents faithfully experimental data measuring the situation of combining concepts. Our model shows how 'interference of concepts' explains the effects of underextension and overextension when two concepts combine to the disjunction of these two concepts. This result supports our earlier hypothesis that human thought has a superposed two-layered structure, one layer consisting of 'classic…
Tokyo Metropolitan University（东京都立大学）；National University of Tainan（台南大学）
In this paper, we propose an AI-FML robotic agent for student learning behavior ontology construction which can be applied in English speaking and listening domain. The AI-FML robotic agent with the ontology contains the perception intelligence, computational intelligence, and cognition intelligence for analyzing student learning behavior. In addition, there are three intelligent agents, including a perception agent, a computational agent, and a cognition agent in the AI-FML robotic agent. We deploy the perception agent and the cognition agent on the robot Kebbi Air. Moreover, the computationa…
Rice University（莱斯大学）；NVIDIA（英伟达）；Baylor College of Medicine（贝勒医学院）；UT Austin（德克萨斯大学奥斯汀分校）；Caltech（加州理工学院）
Humans have an inherent ability to learn novel concepts from only a few samples and generalize these concepts to different situations. Even though today's machine learning models excel with a plethora of training data on standard recognition tasks, a considerable gap exists between machine-level pattern recognition and human-level concept learning. To narrow this gap, the Bongard problems (BPs) were introduced as an inspirational challenge for visual cognition in intelligent systems. Despite new advances in representation learning and learning to learn, BPs remain a daunting challenge for mode…
Identifying, Evaluating, and Mitigating Risks of AI Thought Partnerships
Princeton University（普林斯顿大学）；University of Cambridge（剑桥大学）；Leverhulme Centre for Future Intelligence（勒沃胡姆未来智能中心）；Universitat Politècnica de València（瓦伦西亚理工大学）；The Alan Turing Institute（艾伦·图灵研究所）；NYU（纽约大学）
University of British Columbia（不列颠哥伦比亚大学）；Queensland University of Technology（昆士兰科技大学）
At the core of our uniquely human cognitive abilities is the capacity to see things from different perspectives, or to place them in a new context. We propose that this was made possible by two cognitive transitions. First, the large brain of Homo erectus facilitated the onset of recursive recall: the ability to string thoughts together into a stream of potentially abstract or imaginative thought. This hypothesis is supported by a set of computational models where an artificial society of agents evolved to generate more diverse and valuable cultural outputs under conditions of recursive recall…
A Linguistic Analysis of Spontaneous Thoughts: Investigating Experiences of Déjà Vu, Unexpected Thoughts, and Involuntary Autobiographical Memories
Department of Computer Science, Colorado State University（计算机科学系，科罗拉多州立大学）；Department of Educational Psychology, University of Minnesota（教育心理学系，明尼苏达大学）；Department of Psychology, Colorado State University（心理学系，科罗拉多州立大学）
Query-centric Audio-Visual Cognition Network for Moment Retrieval, Segmentation and Step-Captioning
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）
What lies between `\emph{sensing}' and `\emph{sensibility}'? In other words, what kind of cognitive processes mediate sensing capability, and the formation of sensible impressions ---e.g., abstractions, analogies, hypotheses and theory formation, beliefs and their revision, argument formation--- in domain-specific problem solving, or in regular activities of everyday living, working and simply going around in the environment? How can knowledge and reasoning about such capabilities, as exhibited by humans in particular problem contexts, be used as a model and benchmark for the development of co…
Although artificial intelligence-based perception (AIP) using deep neural networks (DNN) has achieved near human level performance, its well-known limitations are obstacles to the safety assurance needed in autonomous applications. These include vulnerability to adversarial inputs, inability to handle novel inputs and non-interpretability. While research in addressing these limitations is active, in this paper, we argue that a fundamentally different approach is needed to address them. Inspired by dual process models of human cognition, where Type 1 thinking is fast and non-conscious while Typ…
