现代Hopfield检索动力学的保盆地离散化：能量单元、耗散与注意力极限
Basin-Preserving Discretizations of Modern Hopfield Retrieval Dynamics: Energy Cells, Dissipation, and the Attention Limit
浏览论文内容
中文总结 AI 辅助
该研究探讨现代Hopfield网络检索动力学的保盆地离散化，引入能量单元，证明其与连续流、松弛注意力映射及隐式欧拉的包含关系，推导收缩界等结果并经九项数值实验验证。
中文摘要 AI 辅助
现代Hopfield网络的检索动力学是对数求和指数能量的梯度流，而注意力更新是其精确的凸差最小化步长。本研究探讨哪些时间离散化不仅能保持能量衰减与平衡点，还能保持吸引盆。我们引入能量单元——包含一个吸引子且无其他临界点的子水平集的连通分量。主定理表明，低于逃逸能量的每个有限能量单元，同时包含在连续流的盆地、0<θ<2时的每个松弛注意力映射Ψ_θ=(1-θ)id+θ注意力，以及隐式欧拉的唯一性区域内。参数一致的单位曲率上界产生无条件耗散和每个离散步长的单调插值。我们还推导了相距较远模式附近的显式局部收缩界，以及经认证的最优轻微超松弛；表征了保盆地区域之外的近端隧穿与过冲；比较了一阶误差常数；建立了松弛族的标量重参数化的阶障碍；构造了二阶标量-辅助变量格式；并将单元保形性扩展到Bregman几何中的阻尼凸差迭代，包括在有界不对称下的经认证超松弛窗口。九项数值实验验证了这些界与失效机制，在二维盆地实验中，连续与离散检索之间所有观测到的分歧均发生在吸引子特定的数值推断逃逸水平之上。
英文摘要
The retrieval dynamics of a modern Hopfield network is the gradient flow of a log-sum-exp energy, while the attention update is its exact difference-of-convex minimization step. We study which time discretizations preserve not only energy decay and equilibria but also basins of attraction. We introduce energy cells, connected components of sublevel sets containing one attractor and no other critical point. Our main theorem shows that every finite energy cell below the escape energy is contained simultaneously in the basin of the continuous flow, every relaxed attention map $Ψ_θ=(1-θ)\,\mathrm{id}+θ\,\mathrm{attention}$ for $0<θ<2$, and implicit Euler throughout its uniqueness regime. A parameter-uniform unit-curvature majorant yields unconditional dissipation and a monotone interpolation of each discrete step. We also derive explicit local contraction bounds near well-separated patterns, with a certified optimal slight overrelaxation; characterize proximal tunneling and overshoot beyond the preservation regimes; compare first-order error constants; establish an order barrier for scalar reparametrizations of the relaxed family; construct a second-order scalar-auxiliary-variable scheme; and extend cell preservation to damped difference-of-convex iterations in Bregman geometry, including a certified overrelaxed window under bounded asymmetry. Nine numerical campaigns test the bounds and failure mechanisms. In two-dimensional basin experiments, all observed disagreements between continuous and discrete retrieval occur above the attractor-specific numerically inferred escape level.
