Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated "thoughts" reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
State Key Laboratory for Novel Software Technology, Nanjing University, China(新型软件技术国家重点实验室,南京大学,中国);University of California, Merced, USA(加州大学梅德福分校);National Institute of Healthcare Data Science, Nanjing University, China(健康医疗数据科学国立研究所,南京大学,中国)
StreetMath: Study of LLMs' Approximation Behaviors
LuxMuse AI(LuxMuse人工智能公司);Department of Electrical and Computer Engineering, North Carolina State University(北卡罗来纳州立大学电气与计算机工程系);Department of Mathematics, University of Waterloo(滑铁卢大学数学系);Vokram Group(Vokram集团);Department of Computer Science, Saint Louis University(圣路易斯大学计算机科学系)
When Reasoning Goes Astray: Attention Dynamics of Uncontrolled Reasoning
Beijing University of Posts and Telecommunications(北京邮电大学);Wuhan University(武汉大学);JIUTIAN Research(中移九天);Nanyang Technological University(南洋理工大学);Chongqing University of Posts and Telecommunications(重庆邮电大学)