AI 中文总结
研究旨在解构深度网络内部知识，提出HOPE数学框架，将网络压缩从离散域转至希尔伯特空间，统一剪枝与神经元合并为低秩子空间投影，引入宏块逐出涵盖多层结构，通过实验验证该无数据无超参数框架的实用潜力。
AI 中文摘要
深度神经网络编码复杂表示，但解构其内部知识仍是挑战。鉴于学习与压缩的关联，网络压缩是分析此知识的有效视角。然而，标准压缩启发式方法常受规模对称性和架构偏差影响。为解决这些问题，我们引入用于渐进式编码的希尔伯特算子（HOPE），这是一个数学框架，可逐步解构训练网络权重中的表示。HOPE将网络压缩从离散域转移到连续函数的希尔伯特空间。通过将单个神经元建模为秩-1希尔伯特-施密特算子，HOPE将剪枝和神经元合并统一为低秩子空间投影。扩展此公式，HOPE引入宏块逐出，以在相同统一度量下涵盖多层结构，如整个残差路径。这种统一方法能够在不同类型和大小的层之间进行无偏架构决策。HOPE是一个无数据且无超参数的框架。我们在模型压缩和微调方面进行了概念验证实验，以突出我们理论的实际潜力。
英文摘要
Deep neural networks encode complex representations, but deconstructing this internal knowledge remains a challenge. Given the link between learning and compression, network compression offers a promising lens to analyze this knowledge. However, standard compression heuristics often suffer from scale symmetries and architectural biases. To resolve these, we introduce Hilbert Operator for Progressive Encoding (HOPE), a mathematical framework to gradually deconstruct the representations in trained network weights. HOPE shifts network compression from the discrete domain into a Hilbert space of continuous functions. By modeling individual neurons as rank-1 Hilbert-Schmidt operators, HOPE unifies pruning and neuron merging as low-rank subspace projection. Extending this formulation, HOPE introduces macro block eviction to encompass multi-layer structures like entire residual pathways under the same unified metric. This unified approach enables unbiased architectural decisions across layers with different types and sizes. HOPE is a data-free and hyperparameter-free framework. We present proof-of-concept experiments in model compression and fine-tuning to highlight the practical potential of our theory.
