AI 中文总结
该研究分析Sharpness-Aware Minimization(SAM)的平稳性下限,推导不同参数下的复杂度界,引入截断规则并通过数值测试验证其效果,为SAM方法的改进提供了理论与实验依据。
AI 中文摘要
我们针对光滑非凸函数研究一类确定性的Sharpness-Aware Minimization(尖锐度感知最小化,简称SAM)方法,其扰动项为$$ y_k=x_k+\rho\\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^\alpha}, \qquad 0\leq\alpha\leq 1, $$有效半径为$\rho\norm{\nabla f(x_k)}^{1-\alpha}$。对于$0<\alpha\leq1$,我们给出了平稳性水平$(L\rho)^{1/\alpha}$以上的显式复杂度界,一维二次实例可精确达到该水平,表明该界反映了常数参数规则的实际局限;未归一化情形$\alpha=0$需单独处理,要求$L\rho<1$。随后我们引入截断规则,该规则在远离平稳点时与常数$\rho$规则一致,在平稳点附近与梯度成比例,此截断方法满足$\norm{\nabla f(x_k)}\to0$,且具有常规的$O(T^{-1/2})$平稳性界。对二次函数、Rosenbrock函数及五维非凸函数的数值测试,验证了未截断规则的平稳性下限及截断的效果。
英文摘要
We study a deterministic family of sharpness-aware minimization methods for smooth nonconvex functions. The perturbation is $$ y_k=x_k+ρ\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^α}, \qquad 0\leqα\leq 1, $$ so that its effective radius is $ρ\norm{\nabla f(x_k)}^{1-α}$. For $0<α\leq1$, we give an explicit complexity bound above the stationarity level $(Lρ)^{1/α}$. A one-dimensional quadratic example reaches this level exactly, showing that the bound describes a real limitation of the constant-parameter rule. The unnormalized case $α=0$ is treated separately and requires $Lρ<1$. We then introduce a clipped rule which agrees with the constant-$ρ$ rule away from stationary points and becomes proportional to the gradient near them. The clipped method has $\norm{\nabla f(x_k)}\to0$ and the usual $O(T^{-1/2})$ stationarity bound. Numerical tests on a quadratic function, the Rosenbrock function, and a five-dimensional nonconvex function illustrate the stationarity floor of the unclipped rule and the effect of clipping.