Chain-of-thought (CoT) reasoning and its variants have substantially improved the performance of language models on complex reasoning tasks, yet the precise mechanisms by which different strategies facilitate generalization remain poorly understood. While current explanations often point to increased test-time computation or structural guidance, establishing a consistent, quantifiable link between these factors and generalization remains challenging. In this work, we identify intrinsic dimensionality as a quantitative measure for characterizing the effectiveness of reasoning chains. Intrinsic dimensionality quantifies the minimum number of model dimensions needed to reach a given accuracy threshold on a given task. By keeping the model architecture fixed and varying the task formulation through different reasoning strategies, we demonstrate that effective reasoning strategies consistently reduce the intrinsic dimensionality of the task. Validating this on GSM8K with Gemma-3 1B and 4B, we observe a strong inverse correlation between the intrinsic dimensionality of a reasoning strategy and its generalization performance on both in-distribution and out-of-distribution data. Our findings suggest that effective reasoning chains facilitate learning by better compressing the task using fewer parameters, offering a new quantitative metric for analyzing reasoning processes.
CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
UiT - The Arctic University of Norway（UiT - 北极大学）；Integreat - Norwegian Centre for knowledge-driven machine learning（Integreat - 挪威知识驱动机器学习中心）；University of Oslo（奥斯陆大学）
Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter
Vidyasirimedhi Institute of Science and Technology（维达亚希米科技学院）；Kasetsart University（科琼大学）；SCB 10X；King Mongkut’s University of Technology Thonburi（朱拉隆功技术大学泰竹分校）；Department of Computer Engineering Chulalongkorn Univesity（朱拉隆功大学计算机工程系）；Cohere；AI Singapore（AI新加坡）
CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（教育部脑机智能技术重点实验室）
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；AI Foundation and Algorithm Lab（人工智能基金会与算法实验室）；Artificial Intelligence Research Institute (AIRI)（人工智能研究所）；CNRS（法国国家科学研究中心）；Université Paris Cité（巴黎城市大学）；Steklov Institute of Mathematics（斯捷克洛夫数学研究所）
