高容量核关联记忆在稳定性边缘的信息几何自组织
Information Geometric Self-Organization at the Edge of Stability in High-Capacity Kernel Associative Memories
- Aichi Institute of Technology（爱知工业大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究揭示核逻辑回归关联记忆的优化脊为秩1谱坍缩附近的几何奇点，梯度下降通过稳定性边缘现象动态平衡曲率，从而在高曲率边界塑造最优高容量记忆表征。
AI中文摘要：
基于核逻辑回归（KLR）的高容量关联记忆展现出卓越的存储能力和鲁棒性。先前的实证研究确定了一个超参数区域，即“优化脊”，在该区域中吸引子的稳定性达到最大化。然而，该区域的几何本质以及达到该区域所需的优化动态仍不清楚。在本文中，我们研究了KLR训练的Hopfield网络中参数空间的静态几何以及梯度下降（GD）的学习轨迹。利用Hessian矩阵的特征值谱，我们揭示了该脊对应于一个位于秩1谱坍缩附近的相边界，充当一个几何奇点，在该处主曲率被大幅放大。此外，我们证明了学习动态表现出一种由稳定性边缘（EoS）现象驱动的瞬态自稳定行为。网络参数并非寻求平坦区域，而是被驱动向一个状态，在该状态下局部曲率在由学习率决定的稳定性极限附近动态平衡，从而使优化能够在初始不稳定性中存活。我们为秩1渐近坍缩以及控制这种平衡的动态反馈回路提供了解析推导。这些发现表明，最优的高容量记忆表征并非在平坦极小值中形成，而是在几何奇点的高度弯曲边界处被动态塑造。
英文摘要：
High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit exceptional storage capabilities and robustness. Previous empirical studies identified a hyperparameter regime, the "Ridge of Optimization," where attractor stability is maximized. However, the geometric nature of this regime and the optimization dynamics required to reach it have remained unclear. In this paper, we investigate the static geometry of the parameter space and the learning trajectory of Gradient Descent (GD) in KLR-trained Hopfield networks. Using the eigenvalue spectrum of the Hessian, we reveal that the Ridge corresponds to a phase boundary located adjacent to a rank-1 spectral collapse, acting as a geometric singularity where the principal curvature is massively amplified. Furthermore, we demonstrate that the learning dynamics exhibit a transient self-stabilizing behavior driven by the Edge of Stability (EoS) phenomenon. Rather than seeking flat regions, the network parameters are driven toward a state where the local curvature dynamically equilibrates near the stability limit dictated by the learning rate, allowing the optimization to survive the initial instability. We provide analytical derivations for both the rank-1 asymptotic collapse and the dynamic feedback loop governing this equilibration. These findings suggest that optimal, high-capacity memory representations are not formed in flat minima, but are dynamically sculpted at the highly curved boundaries of geometric singularities.
