Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or no finetuning. In contrast, generative reasoning tasks, such as GSM8K and HumanEval\textsuperscript{+}, exhibit substantially weaker recovery. We show that beyond surface-level text degradation, pruning leads to a loss of key algorithmic capabilities, including arithmetic computation and balanced parenthesis generation. Under realistic post-training constraints, using a single 80GB GPU and without access to pretraining-scale data or compute, we evaluate a simple recovery strategy based on supervised finetuning with self-generated responses. This approach recovers up to 90\% of baseline performance on classification tasks, but recovery for generative reasoning remains limited. We further find that this gap persists even under a favorable task-aligned recovery setting, where pruned models are fully finetuned on self-generated GSM8K responses, suggesting that the degradation is not merely due to generic instruction data or parameter-efficient tuning. As complementary evidence, we analyze a depth-pruned model trained with nearly 100B post-pruning tokens and find that deficits persist even on simple arithmetic tasks that do not require multi-step generation. Overall, we characterize practical recovery limits of layer pruning for generative reasoning and provide guidance on when depth reduction is effective under constrained post-training regimes.
Less is More: Towards Green Code Large Language Models via Unified Structural Pruning
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Singapore Management University(新加坡国立管理学院);National University of Defense Technology(国防科技大学);Nantong University(南通大学);Monash University and CSIRO’s Data61(莫纳什大学和CSIRO Data61);Birkbeck, University of London(伦敦大学伯贝克学院)
School of Informatics, Xiamen University, China(厦门大学信息学院);WeChat AI, Tencent Inc, China(腾讯公司微信AI部门);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China(福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室);Shanghai Artificial Intelligence Laboratory, China(上海人工智能实验室)
Parameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
Bernoulli Institute of Mathematics, Computer Science and Artificial Intelligence, University of Groningen(伯努利数学、计算机科学和人工智能研究所,格罗宁根大学);Department of Information and Computing Sciences, Utrecht University(信息与计算科学系,乌得勒支大学)
School of Computer Science and Technology, Beijing Institute of Technology, China(北京理工大学计算机科学与技术学院);School of Computer Science and Technology, Huazhong University of Science and Technology, China(华中科技大学计算机科学与技术学院);Independent, China(独立)
EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning
School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);Department of Data Science, City University of Hong Kong(香港城市大学数据科学系);iFLYTEK Research(iFLYTEK研究院)
Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)
On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning
Department of Computer Science, TU Dortmund University(图恩-多特蒙德大学计算机科学系);Bonn-Aachen International Center for Information Technology, University of Bonn(波恩-亚琛国际信息科技中心,波恩大学);Computer Science School, University of Sheffield(谢菲尔德大学计算机科学学院);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所)
Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
The Hong Kong Polytechnic University(香港理工大学);Singapore Management University(新加坡国立大学);Shandong University(山东大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Towards Reasoning Ability of Small Language Models
Department of Computer Science, Virginia Tech, USA(弗吉尼亚理工大学计算机科学系);Department of Physics, Clarendon Laboratory, University of Oxford, UK(牛津大学物理系,克伦德尔实验室);NVIDIA Corporation(英伟达公司)
From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
Institute of Digital Twin, Eastern Institute of Technology(数字孪生研究院,东部技术研究所);Department of Computing, The Hong Kong Polytechnic University(计算系,香港理工大学);Tencent Hunyuan / AI Lab(腾讯文英/人工智能实验室);Université Paris Dauphine - PSL(巴黎-萨克勒大学)