Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear. We study this question from a probe-time perspective: holding CoT rationales fixed, we test which textual properties matter for the final prediction. Across multiple datasets and model configurations, we find that randomizing the order of rationale sentences has little effect on accuracy, suggesting that the global order of reasoning steps is not the main source of the probe-time benefit. Moreover, even when the words in a rationale are randomly reordered, performance remains well above the no-rationale baseline, indicating that the rationale's words remain useful even without their original order. Restoring only short-range word order further improves performance and brings it substantially closer to full CoT. In most settings, much of this local-order gain is already obtained with three-word windows. Control experiments rule out explicit answer copying, simple lexical cues, generic topical context, and general robustness to shuffling as the main explanations. Mechanistic analyses further show that short-window gains are largely formed in early-to-middle model layers, with answer-relevant evidence concentrated in local text spans. Together, these findings support a local co-occurrence activation (LCA) interpretation: the probe-time benefit of fixed rationales arises mainly from the words they contain and short-range word co-occurrences.
How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；School of Artificial Intelligence and Big Data, Hefei University（合肥大学人工智能与大数据学院）；School of Artificial Intelligence, Beijing Institute of Technology（北京理工大学人工智能学院）；School of Computing and Information, University of Pittsburgh（匹兹堡大学计算机与信息学院）；Center for Biostatistics, The Ohio State University Wexner Medical Center（俄亥俄州立大学韦克斯纳医学中心生物统计中心）
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
National University of Singapore（新加坡国立大学）；Salesforce AI Research（Salesforce AI研究院）；Peking University（北京大学）；Singapore University of Technology and Design（新加坡科技设计大学）
National Center for Artificial Intelligence (CENIA Chile)（国家人工智能中心（智利CENIA））；Millennium Institute for Foundational Research on Data (IMFD Chile)（数据基础研究千年研究所（智利IMFD））；Institute for Mathematical and Computational Engineering, Pontifical Catholic University of Chile（数学与计算工程研究所，智利天主教大学）；Institute of Fundamental Technological Research, Polish Academy of Sciences（基础技术研究所，波兰科学院）
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
CLCG, University of Groningen（格罗宁根大学CLCG）；University of Milano-Bicocca（米兰-布雷拉大学）；University of Trieste（特里耶大学）；Khoury College of Computer Sciences, Northeastern University（东北大学Khoury计算机科学学院）
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Institute for Computational and Mathematical Engineering, Stanford University, Stanford, USA（计算与数学工程研究所，斯坦福大学）；MOE Key Lab of Computational Linguistics, Peking University, Beijing, China（教育部计算语言学重点实验室，北京大学）；Beijing Academy of Artificial Intelligence, Beijing, China（北京人工智能研究院）
