Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing 100084, China（计算机科学与技术系，人工智能研究所，清华-博世联合机器学习中心，THBI实验室，BNRist中心，清华大学，北京100084，中国）；College of AI, Tsinghua University, Beijing 100084, China（人工智能学院，清华大学，北京100084，中国）
Siyuan Zhang, Jialian Li, Yichi Zhang, Xiao Yang, Yinpeng Dong, Hang Su
Large Language Models have achieved remarkable performance on reasoning tasks, motivating research into how this ability evolves during training. Prior work has primarily analyzed this evolution via explicit generation outcomes, treating the reasoning process as a black box and obscuring internal changes. To address this opacity, we introduce a representational perspective to investigate the dynamics of the model's internal states. Through comprehensive experiments across models at various training stages, we discover that post-training yields only limited improvement in static initial representation quality. Furthermore, we reveal that, distinct from non-reasoning tasks, reasoning involves a significant continuous distributional shift in representations during generation. Comparative analysis indicates that post-training empowers models to drive this transition toward a better distribution for task solving. To clarify the relationship between internal states and external outputs, statistical analysis confirms a high correlation between generation correctness and the final representations; while counterfactual experiments identify the semantics of the generated tokens, rather than additional computation during inference or intrinsic parameter differences, as the dominant driver of the transition. Collectively, we offer a novel understanding of the reasoning process and the effect of training on reasoning enhancement, providing valuable insights for future model analysis and optimization.
Beyond representational alignment with brain-guided language models for robust reasoning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University（北京大学通用人工智能国家重点实验室、智能科学与技术学院）；Department of Psychological and Cognitive Sciences, Tsinghua University（清华大学心理与认知科学系）；Microsoft Research Asia（微软亚洲研究院）
Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Davis（加利福尼亚大学戴维斯分校）；New York University（纽约大学）；Shanghai Jiao Tong University（上海交通大学）
Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Models
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；National Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）；Key Laboratory of Machine Perceptron (MOE), Peking University（北京大学机器感知教育部重点实验室）
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
University of Pennsylvania（宾夕法尼亚大学）；New York University（纽约大学）；Indiana University, Bloomington（印第安纳大学，布卢明顿）；Northeastern University（东北大学）；University College London（伦敦大学学院）
Implicit Reasoning in Large Language Models: A Comprehensive Survey
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Jilin University（吉林大学）；The Chinese University of Hong Kong（香港中文大学）；Yale University（耶鲁大学）
From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
School of Information Science, Beijing Language and Culture University（北京语言大学信息科学学院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Ningbo Institute of Digital Twin, Eastern Institute of Technology（宁波数字孪生研究院，东方理工大学）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）
Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Shenzhen Technology University（深圳技术大学）
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
State Key Laboratory of AI Safety,Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
organization= Department of Computer Engineering, Middle East Technical University (METU) , city= Ankara , country= Türkiye；organization= METU-DTX Digital Transformation \& Innovation Centre, METU , city= Ankara , country= Türkiye
