arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2611.06718cs.LG
收藏

基于中心流分析的稳定性边界处的对称性破缺

Symmetry Breaking at the Edge of Stability via Central Flow Analysis

Technical University of Denmark(丹麦技术大学)

Naima Elosegui Borras, Georgios Arvanitidis

AI总结:

该研究基于中心流动力学分析稳定性边界处的对称性破缺,探究梯度流守恒定律的演化,通过三类模型明确了稳定性边界处守恒定律的保留与破缺情况。

摘要:

神经网络中的连续对称性会产生梯度流下的守恒量,例如线性网络和ReLU网络中的层平衡性。有限步梯度下降不一定能保持这些守恒定律,尤其是在稳定性边界(EOS)处,迭代会围绕更平滑的轨迹振荡,这种现象可由中心流动力学描述(Cohen等人,2025)。我们研究该区域的中心流描述下梯度流守恒定律的演化,证明曲率修正会产生由关键参数方向上的振荡协方差决定的有限维漂移。我们通过分析标量两层模型、两层矩阵分解和两层ReLU网络,首次探究了中心流动力学下稳定性边界处哪些守恒定律保留、哪些被破缺,其中已知的守恒定律已被证明是完备的(Marcotte等人,2023)。

英文摘要:

Continuous symmetries in neural networks lead to conserved quantities that are invariant under gradient flow, such as layer balancedness in linear and ReLU networks. Finite-step gradient descent need not preserve these laws, particularly at the edge of stability (EOS), where the iterates oscillate around a smoother trajectory, which can be described by the central flow dynamics (Cohen et al., 2025). We study how gradient flow conservation laws evolve under the central flow description of this regime. We show that the curvature correction induces a finite-dimensional drift determined by the oscillation covariance along critical parameter directions. We thus take the first steps to analyse which conservation laws remain and are broken at the EOS according to the central flow dynamics by analysing a scalar two-layer model, two-layer matrix factorisation, and a two-layer ReLU network, where the known conservation laws are shown to be complete (Marcotte et al., 2023).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑