最优非精确二阶加速在统计相似性下分布式随机优化中的应用

中文摘要 AI 辅助

我们考虑分布式随机凸优化问题，其中固定预算的$N$个独立样本被分配给$m$个工作者。样本平均近似将问题转化为一个正则化的有限和问题，其局部Hessian矩阵具有统计相似性。这使得服务器端局部目标函数的Hessian矩阵可以作为全局目标函数的非精确Hessian矩阵使用，而工作者仅需通信梯度。我们应用了(Chen et al., 2026)的最优加速非精确牛顿外梯度方法，并针对强凸经验问题提出了其分布式重启变体。该方法在$\widetilde O\left(\max\{N^{1/7},m^{1/4}\}\right)$轮通信内达到$N^{-1/2}$阶的统计精度。因此，当$m=N^{4/7}$个工作者时，它需要$\widetilde O\left(N^{1/7}\right)$轮，相比(Agafonov et al., 2021)先前加速三次牛顿构造的$\widetilde O\left(N^{1/6}\right)$，改善了对总样本量的依赖。每次迭代使用两轮梯度聚合，且不需要通信Hessian矩阵。

英文摘要

We consider distributed stochastic convex optimization with a fixed budget of $N$ independent samples split among $m$ workers. Sample average approximation reduces the problem to a regularized finite-sum problem whose local Hessians are statistically similar. This allows the Hessian of the local objective at the server to be used as an inexact Hessian of the global objective, while the workers communicate only gradients. We apply the optimal accelerated inexact Newton extragradient method of (Chen et al., 2026) and propose its distributed restarted variant for the strongly convex empirical problem. The method reaches the statistical accuracy of order $N^{-1/2}$ in $\widetilde O\left(\max\{N^{1/7},m^{1/4}\}\right)$ communication rounds. Hence, with $m=N^{4/7}$ workers, it requires $\widetilde O\left(N^{1/7}\right)$ rounds, improving the dependence on the total sample size from $\widetilde O\left(N^{1/6}\right)$ for the previous accelerated cubic Newton construction of (Agafonov et al., 2021). Each iteration uses two gradient aggregation rounds and does not require Hessian communication.

发表机构

• Moscow Institute of Physics and Technology（莫斯科物理技术学院）
• Innopolis University（因诺波利斯大学）
• Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）

机构由 AI 辅助整理，请以论文原文为准。