NEAT:基于神经元的早期退出用于大规模推理模型
NEAT: Neuron-Based Early Exit for Large Reasoning Models
AI总结:
NEAT通过监控神经元激活动态实现免训练早期退出,减少冗余推理步骤,提升推理效率同时保持解的质量。
摘要:
大规模推理模型(LRMs)常面临过度思考问题,即在正确解被找到后仍生成冗余推理步骤。现有早期推理退出方法主要依赖输出级启发式或训练好的探测模型来跳过冗余步骤,但通常需要额外的rollout计算或外部标记数据集。本文提出NEAT,一种基于神经元的早期推理退出框架,通过监控神经元层面的激活动态实现免训练的早期退出,无需引入额外的测试时间计算。NEAT识别与退出相关的神经元并跟踪其激活模式,在推理过程中动态触发早期退出或抑制反思,从而减少不必要的推理同时保持解的质量。在六个不同规模和架构的模型上四个推理基准测试中,NEAT在每个模型上平均实现22%至28%的平均token减少,同时保持准确性。
英文摘要:
Large Reasoning Models (LRMs) often suffer from \emph{overthinking}, a phenomenon in which redundant reasoning steps are generated after a correct solution has already been reached. Existing early reasoning exit methods primarily rely on output-level heuristics or trained probing models to skip redundant reasoning steps, thereby mitigating overthinking. However, these approaches typically require additional rollout computation or externally labeled datasets. In this paper, we propose \textbf{NEAT}, a \textbf{N}euron-based \textbf{E}arly re\textbf{A}soning exi\textbf{T} framework that monitors neuron-level activation dynamics to enable training-free early exits, without introducing additional test-time computation. NEAT identifies exit-associated neurons and tracks their activation patterns during reasoning to dynamically trigger early exit or suppress reflection, thereby reducing unnecessary reasoning while preserving solution quality. Experiments on four reasoning benchmarks across six models with different scales and architectures show that, for each model, NEAT achieves an average token reduction of 22\% to 28\% when averaged over the four benchmarks, while maintaining accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。