光滑凸优化中长步长方法与加速方法的经验与计算机辅助鲁棒性分析
Empirical and computer-aided robustness analysis of long-step and accelerated methods in smooth convex optimization
AI总结：
本文通过性能估计方法从理论与实验两方面分析一阶优化方法在梯度相对不精确时的鲁棒性，发现长步长与加速方法理论上不鲁棒，提出缩短因子改进保证，并验证加速方法实际鲁棒性优于预期。
摘要：
本文利用性能估计方法，从经验和理论两个层面评估了不同一阶优化方法在其梯度计算受到相对不精确性影响时的鲁棒性。相对不精确性出现于例如使用更少比特信息压缩梯度时，这在大规模GPU问题中很常见。本文分析了三大类方法：固定步长梯度下降、长步长方法和加速方法。首先从理论上证明后两类方法对不精确性不具有鲁棒性。随后引入一种半启发式的缩短因子以改进其理论保证。所有方法随后在一个具体的不精确问题上进行测试，采用两种不同类型的相对不精确性，观察到两种加速方法都比预期鲁棒得多，且缩短因子显著帮助了长步长方法。最终，所有缩短后的方法即使在这种不精确设置下也表现出良好的前景。
英文摘要：
This work assesses both empirically and theoretically, using the performance estimation methodology, how robust different first-order optimization methods are when subject to relative inexactness in their gradient computations. Relative inexactness occurs, for example, when compressing the gradient using fewer bits of information, which happens when dealing with large-scale problems on GPUs. Three major families of methods are analyzed: constant step gradient descent, long-step methods, and accelerated methods. The latter two are first shown to be theoretically not robust to inexactness. Then, a semi-heuristic shortening factor is introduced to improve their theoretical guarantees. All methods are subsequently tested on a concrete inexact problem, with two different types of relative inexactness, and it is observed that both accelerated methods are much more robust than expected, and that the shortening factor significantly helps the long-step methods. In the end, all shortened methods appear to be promising, even in this inexact setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
