Although neural models have performed impressively well on various tasks such as image recognition and question answering, their reasoning ability has been measured in only few studies. In this work, we focus on spatial reasoning and explore the spatial understanding of neural models. First, we describe the following two spatial reasoning IQ tests: rotation and shape composition. Using well-defined rules, we constructed datasets that consist of various complexity levels. We designed a variety of experiments in terms of generalization, and evaluated six different baseline models on the newly generated datasets. We provide an analysis of the results and factors that affect the generalization abilities of models. Also, we analyze how neural models solve spatial reasoning tests with visual aids. Our findings would provide valuable insights into understanding a machine and the difference between a machine and human.
Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
Fudan University（复旦大学）；Alibaba Group Holding Limited（阿里巴巴集团控股有限公司）；Future Living Lab of Alibaba（阿里巴巴未来生活实验室）；University of Southern California（南加州大学）；Shanghai Innovation Institute（上海创新研究院）
TInR: Exploring Tool-Internalized Reasoning in Large Language Models
The Hong Kong Polytechnic University（香港理工大学）；Southeast University（东南大学）；University of Edinburgh（爱丁堡大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing
Beihang University, Beijing, China（北京航空航天大学）；University of Liverpool, Liverpool, UK（利物浦大学）；Institute of Acoustics, Chinese Academy of Sciences, Beijing, China（中国科学院声学研究所）
Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning
The Hong Kong Polytechnic University（香港理工大学）；University of California, Santa Barbara（加州大学圣芭芭拉分校）；Peking University（北京大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The University of Hong Kong（香港大学）
The Hong Kong Polytechnic University（香港理工大学）；Southern University of Science and Technology（南方科技大学）；Tsinghua University（清华大学）；Reallm Labs（Reallm实验室）；Wuhan University（武汉大学）；The University of Hong Kong（香港大学）
School of Informatics, Xiamen University, China（厦门大学信息学院）；WeChat AI, Tencent Inc, China（腾讯公司微信AI部门）；Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China（福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室）；Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）
Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability
State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统管理与控制国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Exploring the Necessity of Reasoning in LLM-based Agent Scenarios
Huazhong University of Science and Technology（华中科技大学）；Universiti Malaya（马来亚大学）；Wuhan University of Technology（武汉理工大学）；Duke University（杜克大学）；Lehigh University（理海大学）
Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理研究重点实验室）；Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（教育部新一代智能搜索与推荐工程研究中心）；Tencent Jarvis Lab（腾讯杰维斯实验室）；Medical Artificial Intelligence Lab, Westlake University（西湖大学医学人工智能实验室）
Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
School of Computer Science and Technology, Xi'an Jiaotong University（西安交通大学计算机科学与技术学院）；Department of Computer Science, The University of Manchester（曼彻斯特大学计算机科学系）；Hunan University（湖南大学）；National University of Singapore（新加坡国立大学）
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges
Shanghai AI Laboratory（上海人工智能实验室）；Wuhan University（武汉大学）；Renmin University of China（中国人民大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Jiao Tong University（上海交通大学）
An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Innovation Center for Processor Technologies（上海处理器技术创新中心）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）
Reasoning or Rambling? Exploring the Effect of Thinking on Agent Persuasion
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；National University of Singapore（新加坡国立大学）；Inner Mongolia Research Institute, Shanghai Jiao Tong University（上海交通大学内蒙古研究院）
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院 Gallagher 分校）；Baidu Inc., Beijing, China（百度公司，北京，中国）；Carnegie Mellon University（卡内基梅隆大学）
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Shanghai Key Laboratory of Intelligent Information Processing, Fudan University（复旦大学上海智能信息处理重点实验室）
