发表机构
LMU Munich; Munich Center for Machine Learning (MCML); Goodfire AI; DLR-German Aerospace Center; University of Tromsø（慕尼黑大学; 慕尼黑机器学习中心; 古德菲尔人工智能公司; 德国航空航天中心; 特罗姆瑟大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对模型压缩问题，提出用行为恢复秩量化功能退化，发现结构与幅度剪枝在任务饱和后仍保留更多自由度，揭示功能冗余分布于参数方向。
AI 中文摘要
现代机器学习的核心问题是，训练好的模型在不改变其行为的前提下能被压缩多少，以降低部署所需的内存、计算和能耗。为研究这一问题，我们通过行为恢复秩量化功能退化，该指标定义为恢复训练模型性能所需的前导行为-海森本征方向的数量。以行为恢复秩作为压缩的几何基准，我们发现结构剪枝和幅度剪枝在任务饱和后仍保留更多自由度。这一差距表明，功能冗余分布在参数方向上，无法被单个权重或神经元所暴露。
英文摘要
A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute and energy required to deploy it. To study this, we quantify functional degeneracy through the behavioral recovery rank, defined as the number of leading behavioral-Hessian eigendirections required to recover a trained model's performance. Using the behavioral recovery rank as a geometric benchmark for compression, we find that structural and magnitude pruning retain more degrees of freedom, even after the task is saturated. This gap suggests that functional redundancy is distributed across parameter directions and is not exposed by individual weights or neurons.
