发表机构
Levich Institute; City College of New York; Biomedical Engineering Department, City College of New York（莱维奇研究所; 纽约城市学院; 纽约城市学院生物医学工程系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究发现深度神经网络学习会涌现纤维化等图论对称性，利用这些对称性可压缩模型至原规模17%，可控破缺对称性还能提升持续学习性能，为AI系统提供了新理论基础。
AI 中文摘要
人工神经网络常被视为强大却不透明的黑箱。本文证明，深度神经网络的学习过程会产生图论中称为纤维化和覆盖的局部对称性。我们证明覆盖对称性是随机梯度下降的稳定吸引子。与该理论一致，我们报告在多层、卷积、循环及Transformer网络等主流架构中均出现了覆盖对称性。利用这些对称性可实现大幅模型压缩，将网络缩小至原始规模的17%而不损失性能。此外，对覆盖对称性的可控破缺可克服可塑性丧失问题，在持续学习中达到了当前最优性能。这些理论结果为基于对称性的AI系统提供了新基础，该系统可将黑箱转换为可解释的有色图，并实现更高效的推理与终身学习。
英文摘要
Artificial neural networks are often regarded as powerful yet opaque black boxes. Here, we demonstrate that learning in deep neural networks generates local symmetries known in graph theory as fibrations and coverings. We prove that covering symmetries are stable attractors of stochastic gradient descent. Consistent with this theory, we report the emergence of covering symmetries across major network architectures, including multilayer, convolutional, recurrent, and transformer networks. Exploiting these symmetries enables drastic model compression - reducing networks to 17% of their original size without sacrificing performance. Furthermore, controlled breaking of covering symmetry overcomes the loss of plasticity, achieving state-of-the-art performance in continual learning. The theoretical results provide a new foundation for AI systems based on symmetries that convert black boxes into interpretable colored graphs and enable more efficient inference and lifelong learning.
Comments11 Figures
