发表机构
University of Illinois Chicago（伊利诺伊大学芝加哥分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文从统一视角研究神经网络计算缩减，证明一次性幅度剪枝的集中定理，并引入条件感知器早退，其泛化误差随计算差距幂次衰减，扩展至深度网络并验证标度律。
AI 中文摘要
我们通过统一的局部与全量计算视角研究神经网络中的计算缩减，该视角由静态机制下的一次性幅度剪枝和自适应机制下的早退所捕获。在渐近单神经元模型中，我们证明了一次性幅度剪枝的集中定理，并给出了显式速率。我们还引入了用于早退的条件感知器，并证明其超额泛化误差随计算差距的幂次衰减，且该指数在局部与全量计算之间的对齐趋于1时增长至无穷大。随后，我们将分析扩展到深度网络，刻画了剪枝引起的失真如何随深度累积，并在神经网络高斯过程模型下推导了冻结骨干早退的相应计算-精度权衡。数值模拟验证了预测的标度律。
英文摘要
We study compute reduction in neural networks through a unified partial versus full computation view, captured by one-shot magnitude pruning in the static regime and early exit in the adaptive regime. In an asymptotic single-neuron model, we prove a concentration theorem for one-shot magnitude pruning with explicit rates. We also introduce the conditional perceptron for early exit and show that its excess generalization error decays as a power of the compute gap, with an exponent that grows to infinity as the alignment between partial and full computations tends to one. We then extend the analysis to deep networks, characterizing how pruning-induced distortions accumulate with depth and deriving a corresponding compute-accuracy tradeoff for frozen-backbone early exit under a neural network Gaussian process model. Numerical simulations corroborate the predicted scaling laws.
CommentsPublished in ICML 2026
