SHANG++: 在乘性噪声下的鲁棒随机加速
SHANG++: Robust Stochastic Acceleration under Multiplicative Noise
AI总结：
本文提出SHANG++方法，通过离散化Hessian驱动的Nesterov加速梯度流，在乘性噪声条件下提升稳定性与收敛速度，实验显示其在鲁棒性和效率上优于现有方法。
摘要：
在乘性噪声缩放（MNS）条件下，原始Nesterov加速方法被证明对噪声敏感，当梯度噪声主导信号时可能发散。本文开发了两种加速随机梯度下降方法，通过离散化Hessian驱动的Nesterov加速梯度流。首先推导出SHANG，一种直接的Gauss-Seidel型离散化方法，已在MNS下提升了稳定性。然后引入SHANG++，添加阻尼修正，实现更快的收敛速度和更强的噪声鲁棒性。我们为凸和强凸目标函数在MNS下建立了收敛保证，并提供了显式参数选择。在实验中，SHANG++在凸问题和深度学习应用中表现稳定。在专门的噪声实验中，ResNet-34上单个超参数配置在噪声环境下达到接近无噪声设置的准确率。所有实验中，SHANG++在鲁棒性和效率上均优于现有加速方法，且参数敏感性极低。
英文摘要：
Under the multiplicative noise scaling (MNS) condition, original Nesterov acceleration is provably sensitive to noise and may diverge when gradient noise overwhelms the signal. In this paper, we develop two accelerated stochastic gradient descent methods by discretizing the Hessian-driven Nesterov accelerated gradient flow. We first derive SHANG, a direct semi-implicit discretization that already improves stability under MNS. We then introduce SHANG++, which adds a damping correction and achieves faster convergence with greater noise robustness. We establish convergence guarantees for both convex and strongly convex objectives under MNS, together with explicit parameter choices. In our experiments, SHANG++ performs consistently well across convex problems and applications in deep learning. In a dedicated noise experiment on ResNet-34, a single hyperparameter configuration maintains accuracy within one percentage point of the noise-free setting. Across all experiments, SHANG++ outperforms existing accelerated methods in robustness and efficiency, with minimal parameter sensitivity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
