1. We have updated the results for DeepSeek-R1, and all of our original conclusions remain valid. 2. Our proposed Tip approach remains effective in Best-of-N scenarios (e.g., self-consistency and Laconic Decoding) when built on DeepSeek-R1
Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable abilities in complex reasoning tasks by scaling test-time compute and exhibiting human-like deep thinking. However, we identify a phenomenon we term underthinking, where o1-like LLMs frequently switch between different reasoning thoughts without sufficiently exploring promising paths to reach a correct solution. This behavior leads to inadequate depth of reasoning and decreased performance, particularly on challenging mathematical problems. To systematically analyze this issue, we conduct experiments on three challenging test sets and two representative open-source o1-like models, revealing that frequent thought switching correlates with incorrect responses. We introduce a novel metric to quantify underthinking by measuring token efficiency in incorrect answers. To address underthinking, we propose a decoding strategy with thought switching penalty TIP that discourages premature transitions between thoughts, encouraging deeper exploration of each reasoning path. Experimental results demonstrate that our approach improves accuracy across challenging datasets without requiring model fine-tuning. Our findings contribute to understanding reasoning inefficiencies in o1-like LLMs and offer a practical solution to enhance their problem-solving capabilities.
Fudan University(复旦大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Tongji University(同济大学);Shanghai Jiaotong University(上海交通大学);Nankai University(南开大学);Hong Kong Polytechnic University(香港理工大学);University of California, Merced(加州大学默塞德分校);University of Science and Technology of China(中国科学技术大学)
Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges
School of Computing and Electrical Engineering(计算与电子工程学院);Indian Institute of Technology (IIT) Mandi(印度理工学院(IIT)曼迪);London U.K.(伦敦英国);Warwick Manufacturing Group U.K.(沃里克制造集团英国)
Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective
Department of Computer Science and Engineering, The Pennsylvania State University(宾夕法尼亚州立大学计算机科学与工程系);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
KAUST(阿卜杜拉国王科技大学);Soochow University(苏州大学);University of Southampton(南安普顿大学);Tongji University(同济大学);Universität Hamburg(汉堡大学);Huazhong University of Science and Technology(华中科技大学)