发表机构
Faculty of Information Technology, Beijing University of Technology（北京工业大学信息技术学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对标准SAM在参数块间分配扰动预算不合理的问题，提出GEAR-SAM，以平方块梯度的EMA为灵敏度信号，通过闭式约束优化分配预算，实验表明其在多种任务中提升了泛化和鲁棒性，还提供了动态优化视图。
AI 中文摘要
锐度感知最小化（SAM）通过最小化局部参数邻域中的最坏情况损失来提高泛化能力。标准SAM根据瞬时小批量梯度范数在参数块之间隐式分配其全局扰动预算。这种分配可能有噪声，且可能无法反映块在整个训练过程中积累的灵敏度。我们提出了梯度能量自适应半径SAM（GEAR-SAM），它将平方块梯度的指数移动平均值（EMA）作为轻量级、与曲率相关的灵敏度信号，并通过闭式约束优化来分配固定的SAM预算。GEAR-SAM保留了全局SAM半径，不需要海森向量积或显式的费舍尔估计，并且除了SAM之外只添加标量状态。在图像分类、迁移学习、噪声标签学习和分区研究上的实验表明，在各种架构和任务中，泛化能力和鲁棒性都得到了提高。更广泛地说，GEAR-SAM提供了一个动态锐度感知优化视图：随着功能网络块的灵敏度在训练过程中演变，固定的扰动预算应该重新分配。
英文摘要
Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood. Standard SAM implicitly allocates its global perturbation budget across parameter blocks according to instantaneous minibatch gradient norms. Such an allocation can be noisy and may not reflect the sensitivity that blocks accumulate throughout training. We propose Gradient-Energy Adaptive Radius SAM (GEAR-SAM), which maintains an exponential moving average (EMA) of squared block gradients as a lightweight, curvature-related sensitivity signal and allocates the fixed SAM budget through a closed-form constrained optimization. GEAR-SAM preserves the global SAM radius, requires no Hessian-vector products or explicit Fisher estimation, and adds only scalar state beyond SAM. Experiments on image classification, transfer learning, noisy-label learning, and partition studies demonstrate improved generalization and robustness across architectures and tasks. More broadly, GEAR-SAM provides a dynamic view of sharpness-aware optimization: a fixed perturbation budget should be redistributed as the sensitivity of functional network blocks evolves during training.
