Cognitive textual and visual reasoning tasks, including puzzles, series, and analogies, demand the ability to quickly reason, decipher, and evaluate patterns both textually and spatially. Due to extensive training on vast amounts of human-curated data, LLMs and VLMs excel in common-sense reasoning tasks, however still struggle with more complex reasoning that demands deeper cognitive understanding. We introduce NTSEBench, a new dataset designed to evaluate cognitive multi-modal reasoning and problem-solving skills of large models. The dataset contains 2728 multiple-choice questions, accompanied by a total of 4,642 images, categorized into 26 different types. These questions are drawn from the nationwide NTSE examination in India and feature a mix of visual and textual general aptitude challenges, designed to assess intelligence and critical thinking skills beyond mere rote learning. We establish baselines on the dataset using state-of-the-art LLMs and VLMs. To facilitate a comparison between open source and propriety models, we propose four distinct modeling strategies to handle different modalities -- text and images -- in the dataset instances.
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
Seoul National University（首尔国立大学）；Human-Computer Interaction Institute, Carnegie Mellon University（人机交互研究所，卡内基梅隆大学）；Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）
Hán Dān Xué Bù (Mimicry) or Qīng Chū Yú Lán (Mastery)? A Cognitive Perspective on Reasoning Distillation in Large Language Models
School of Philosophy, Anhui University（安徽大学哲学学院）；Institute of Neuroscience, Chinese Academy of Sciences（中国科学院神经科学研究所）；Faculty of Education, University of Cambridge（剑桥大学教育学院）；School of Foreign Studies, South China Normal University（华南师范大学外语学院）；Division of Psychology and Language Sciences, University College London（伦敦大学学院心理学与语言科学系）
CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Models
School of Computer Science and Engineering, Beihang University, China（北京航空航天大学计算机科学与工程学院）；Meituan Inc.（美团公司）；Beijing Institute for General Artificial Intelligence, China（北京通用人工智能研究院）
CLiViS: Unleashing Cognitive Map through Linguistic-Visual Synergy for Embodied Visual Reasoning
School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）；King Abdullah University of Science and Technology（科廷大学）；Fudan University（复旦大学）
11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
Microsoft Research（微软研究院）；Language Technology Lab, University of Cambridge（语言技术实验室，剑桥大学）；Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）；Department of Oncology, University of Cambridge（癌症部门，剑桥大学）；Leverhulme Centre for the Future of Intelligence, University of Cambridge（未来智能中心，剑桥大学）；VRAIN, Universitat Politècnica de València（VRAIN，巴塞罗那理工大学）
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
Rutgers University（罗格斯大学）；South China Agricultural University（华南农业大学）；Columbia University（哥伦比亚大学）；Fenz.AI；QuantaAlpha；Adobe；Santa Clara University（圣克拉拉大学）；City University of Hong Kong（香港城市大学）
Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）；Ant Group（蚂蚁集团）
Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
South China University of Technology（南方科技大学）；Pazhou Laboratory（琶洲实验室）；Peng Cheng Laboratory（鹏城实验室）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
MedCoG: Maximizing LLM Inference Density in Medical Reasoning via Meta-Cognitive Regulation
College of Computer Science, VCIP, DISSec Center, Nankai University, China 300350（南开大学计算机学院、VCIP、DISSec中心、中国300350）；Generative AI Lab, College of Computing and Data Science（生成式人工智能实验室、计算与数据科学学院）；Nanyang Technological University, Singapore 639798（南洋理工大学，新加坡639798）
Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding
Centre for Vision, Speech and Signal Processing (CVSSP)（视觉、语音与信号处理中心）；University of Surrey（萨里大学）；Surrey Institute for People-Centred AI (PAI)（萨里以人为本人工智能研究所）
Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学 龙城人工智能学院）；South China Agricultural University（华南农业大学）；South China Normal University（华南师范大学）；Monash University（莫纳什大学）；Jishou University（吉首大学）；Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）
