深度神经网络中选择性光谱对齐的对易子框架
A Commutator Framework for Selective Spectral Alignment in Deep Neural Networks
- Uppsala University（乌普萨拉大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
该研究提出对易子框架量化深度神经网络中特征几何的不兼容性，通过逐层分解揭示光谱对齐的层与尺度依赖性，经实验验证其由传输、抵消等因素主导，非训练普遍结果。
中文摘要 AI 辅助
我们提出了一种有限宽度几何框架，用于描述深度神经网络中学习到的特征几何如何被组织、传输和选择性对齐。权重生成的协方差、门控以及反向灵敏度之间的不兼容性通过三类对易子量化：门控与协方差之间、灵敏度与协方差之间，以及平均梯度外积（AGOPs）与神经特征矩阵（NFMs）之间。一个精确的逐层恒等式将灵敏度-协方差对易子分解为四个来源：下游传输、相邻层不平衡、逐点灵敏度波动以及非线性门控-协方差相互作用。AGOP-NFM对易子是内部对易子的奇异值加权传输，解释了为何仅观测到的特征侧对齐无法确定其产生的内部几何。缓冲局部能量可解决分离协方差子空间之间的混合问题。我们建立了谱间隙、投影器演化和稳定性估计，并构建了条件李雅普诺夫原理，该原理在明确的几何误差边界或固有阻尼假设下可产生衰减。这些准则无法仅从梯度流推导得出，且阐明了为何风险降低不一定意味着对易子坍缩。解析示例和数值实验展示了谱与激活几何的分解、瞬态增长以及非零来源之间的抵消。在测试的有限时间范围内，由负传输-不平衡相互作用主导的抵消在深度、宽度及两个回归基准中均持续存在。因此，光谱对齐是一种依赖于层和尺度的兼容现象，由传输、相互作用、抵消及可能的阻尼所支配，而非训练的普遍结果。
英文摘要
We develop a finite-width geometric framework describing how learned feature geometries are organized, transported, and selectively aligned in deep neural networks. Incompatibility among weight-generated covariance, gates, and backward sensitivities is quantified through three families of commutators: between gates and covariance, between sensitivities and covariance, and between average gradient outer products (AGOPs) and neural feature matrices (NFMs). An exact layerwise identity decomposes the sensitivity-covariance commutator into four sources: downstream transport, adjacent-layer imbalance, pointwise sensitivity fluctuations, and nonlinear gate-covariance interactions. The AGOP-NFM commutator is a singular-value-weighted transport of the internal commutator, explaining why observed feature-side alignment alone does not determine the internal geometry from which it emerges. Buffered localized energies resolve mixing between separated covariance subspaces. We establish spectral-gap, projector-evolution, and stabilization estimates, and formulate conditional Lyapunov principles that yield decay under explicit geometric error-bound or intrinsic-damping assumptions. These criteria do not follow from gradient flow alone and clarify why risk reduction need not imply commutator collapse. Analytic examples and numerical experiments exhibit factorization of spectral and activation geometry, transient growth, and cancellation among nonzero sources. In tested finite-time regimes, cancellation dominated by a negative transport-imbalance interaction persists across depths, widths, and two regression benchmarks. Spectral alignment therefore appears as a layer- and scale-dependent compatibility phenomenon governed by transport, interaction, cancellation, and possible damping, rather than a universal consequence of training.
