SRKCD：一种求解随机最优化的稳定化龙格-库塔法
SRKCD: a stabilized Runge-Kutta method for stochastic optimization
摘要：
介绍了一类基于Runge-Kutta-Chebyshev（RKC）格式的随机最优化方法。RKC方法是一种显式方法，最初设计用于求解刚性常微分方程，通过确保其稳定区域具有最大尺寸。在优化上下文中，与流行的随机梯度下降法相比，这允许更大的步长（学习速率）和更好的鲁棒性。我们的主要贡献是基本上所有随机龙格-库塔最优化方法的收敛性证明。这表明在强凸性和Lipschitz连续梯度的标准假设下，具有最优次线性速率的期望收敛。对于非凸目标，我们在梯度的期望下收敛到零。证明需要一定的自然条件的龙格-库塔系数，我们进一步证明，RKC格式满足这些。最后，我们说明了改进的稳定性的方法在实践中进行数值实验的小规模的测试示例和机器学习中的图像分类应用程序所产生的问题。
英文摘要：
We introduce a family of stochastic optimization methods based on the Runge-Kutta-Chebyshev (RKC) schemes. The RKC methods are explicit methods originally designed for solving stiff ordinary differential equations by ensuring that their stability regions are of maximal size.In the optimization context, this allows for larger step sizes (learning rates) and better robustness compared to e.g. the popular stochastic gradient descent method. Our main contribution is a convergence proof for essentially all stochastic Runge-Kutta optimization methods. This shows convergence in expectation with an optimal sublinear rate under standard assumptions of strong convexity and Lipschitz-continuous gradients. For non-convex objectives, we get convergence to zero in expectation of the gradients. The proof requires certain natural conditions on the Runge-Kutta coefficients, and we further demonstrate that the RKC schemes satisfy these. Finally, we illustrate the improved stability properties of the methods in practice by performing numerical experiments on both a small-scale test example and on a problem arising from an image classification application in machine learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
