AI 中文总结
本文针对$(H_0,H_1)$-光滑凸优化，结合Nesterov型加速梯度等技术提出全梯度与随机坐标加速算法，给出迭代复杂度并经实验验证，为该类优化提供了首个加速理论保证。
AI 中文摘要
我们针对凸$(H_0,H_1)$-光滑优化问题开发了加速算法，其中$\nabla^2 f(x)$的范数不超过$H_0+H_1(f(x)-f^*)$。该类优化问题是标准光滑优化的推广，包含$(L_0,L_1)$-光滑类。结合Nesterov型加速梯度方案、小维松弛和相位重启，我们得到了一个全梯度方法，其迭代复杂度为$\tilde O(\tilde{\frac{\text{sqrt}(H_0 \tilde R^2/\text{varepsilon})}{\text{sqrt}(H_1 \tilde R^2)} \times \text{log}(F_0/\text{varepsilon})})$。我们将该方法扩展到随机坐标优化，得到了采用均匀采样的坐标方法，其迭代复杂度带有标准因子$d$，以及采用非均匀采样的坐标方法，其迭代复杂度由$S_{1/2}^{(j)}=\text{sum}_i\text{sqrt}(H_{j,i})$决定。据我们所知，这些结果为该凸类提供了首个加速全梯度和坐标优化的理论保证。我们还给出了实用实现建议。实验证实了预测的加速效果、非均匀采样的收益以及不精确松弛的可行性。
英文摘要
We develop accelerated algorithms for convex $(H_0,H_1)$-smooth optimization, where $\|\nabla^2 f(x)\|\le H_0+H_1(f(x)-f^*)$. This class generalizes standard smoothness and contains the $(L_0,L_1)$-smooth class. Combining a Nesterov-type accelerated gradient scheme with small-dimensional relaxation and phase restarts, we obtain a full-gradient method with iteration complexity $\widetilde O(\sqrt{H_0\widetilde R^2/\varepsilon}+\sqrt{H_1\widetilde R^2}\log(F_0/\varepsilon))$. We extend the same approach to randomized coordinate optimization, obtaining a coordinate method with uniform sampling whose iteration complexity carries the standard factor $d$, and a coordinate method with non-uniform sampling whose iteration complexity is governed by $S_{1/2}^{(j)}=\sum_i\sqrt{H_{j,i}}$. These results provide, to our knowledge, the first accelerated full-gradient and coordinate guarantees for this convex class. We also provide practical implementation recommendations. Experiments confirm the predicted acceleration, gains from non-uniform sampling, and the viability of inexact relaxation.
CommentsPreprint
