arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.05643cs.CL
收藏

基于图的链式推理剪枝:减少推理LLM中冗余反射

Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs

School of Geosciences and Info-Physics, Central South University(地质科学与信息物理学院,中南大学);Baidu Inc.(百度公司)

Hongyuan Yuan, Xinran He, Run Shao, Bolei He, Xianwei Xue, Mengke Chen, Qiutong Pan, Haiwei Wang, Haifeng Li

AI总结:

本文提出基于图的链式推理优化框架,通过剪枝策略减少推理过程中冗余反射,提升效率与准确性。

摘要:

通过强化学习扩展链式推理(CoT)已广泛用于增强LLM的推理能力。然而,由于奖励信号稀疏,它也可能导致不可取的思维模式,如过度思考,即生成冗余的中间推理内容。本文认为,冗余的主要来源是低效的反思,通常表现为两种问题模式:无 discriminate反思,即模型在推理过程中进行广泛的低影响检查;以及重复反思,即模型重复验证已确立的结论。为了解决这个问题,我们引入了基于图的Co T优化框架。具体来说,我们将每个线性Co T转换为具有显式依赖边的有向无环图(DAG),并设计了双剪枝策略:分支级剪枝删除贡献弱的反思分支,而深度级剪枝消除后期的重复验证。我们通过三阶段流程进行蒸馏:(1)SFT在修剪的简洁轨迹上初始化策略,(2)DPO偏好正确但冗余较少的轨迹,(3)GRPO带有长度惩罚共同优化答案的正确性和效率。实验表明,我们的方法在保持或提高准确性的同时,将平均推理令牌减少了42%。

英文摘要:

Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42\% while maintaining or improving accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑