发表机构
School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对一维二次函数设定，分析未修正Adam的稳定边缘动力学，证明其向特定稳定阈值的恢复倾向，同时指出其寻边机制失效的情况，为Adam的稳定边缘现象提供了动力学解释并揭示其局限性。
AI 中文摘要
Adam的稳定边缘（EoS）现象已被广泛观察，但其潜在动力学机制尚未完全明晰。我们在一维二次函数这一干净设定下研究未修正Adam，其中恒定曲率隔离了优化器诱导的EoS背后的动力学。我们刻画了参数空间上的所得动力学，在广泛范围内证明Adam表现出向其冻结稳定阈值2(1+β₁)/[η(1−β₁)]的恢复倾向；还识别出该寻边机制失效的设定，包括严格亚临界周期轨道及收敛到最优值却始终保持一致超临界的特殊调优轨迹。这些结果在无演化损失几何的设定下为Adam的EoS提供了具体动力学解释，同时也暴露了其局限性。
英文摘要
The edge-of-stability (EoS) phenomenon of full-batch Adam has been widely observed, yet its underlying dynamical mechanism remains poorly understood. In this paper, we identify Adam's second-moment adaptation as a negative-feedback mechanism that drives the dynamics toward the stability boundary. We characterize this mechanism through the *active curvature*, namely, the preconditioned curvature along the preconditioned gradient direction, and establish rigorous characterizations in progressively richer settings: rank-one quadratics with momentum, diagonal quadratics, on which the active curvature separates from the sharpness, and general objectives. Importantly, the mechanism predicts *gradient reversal* of full-batch Adam near the edge: consecutive gradients repeatedly point in nearly opposite directions, as we observe across fully connected networks, ResNets, ViTs, LSTMs, GPT-2 medium, and Adam-family optimizers. Consistent with this picture, averaging iterates suppresses these fast oscillations and produces smoother and lower loss curves. Together, these results provide an important first step towards fully understanding the dynamical behavior of Adam's EoS through active curvature and gradient reversal.
