arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.10706math.OCcs.DCcs.LG
收藏

相似条件下的分布式鞍点问题

Distributed Saddle-Point Problems Under Similarity

Moscow Institute of Physics and Technology(莫斯科物理技术学院);ISP RAS(俄罗斯科学院信息传输问题研究所);Research Center for Trusted Artificial Intelligence(可信人工智能研究中心);MIPT(莫斯科物理技术学院);HSE University(高等经济大学);Yandex(扬德克斯);Purdue University(普渡大学)

Aleksandr Beznosikov, Gesualdo Scutari, Alexander Rogozin, Alexander Gasnikov

摘要:

本文研究了两类网络上的(强)凸-(强)凹鞍点问题(SPP)的求解方法--主/工(中心化)网络和网状(分散化)网络。由于统计数据相似性或其他原因,假设每个节点处的局部函数是相似的。我们建立一个相当一般的一类算法解决SPP的复杂性较低的界限。我们发现,一个给定的次优$ε>0$是实现在主/工人网络在$Ω\big(Δ\cdot δ/μ\cdot \log (1/\varepsilon)\big)$轮的通信,其中$δ>0$措施的相似程度的本地功能,$μ$是他们的强凸常数,和$Δ$是网络的直径。网状网络上的较低通信复杂度界限读作$Ω\big(1/{\sqrtρ} \cdot δ/μ\cdot\log (1/\varepsilon)\big)$,其中$ρ$是用于相邻节点之间的通信的流言矩阵的(归一化)本征间隙。然后,我们提出了算法匹配的下界在任何类型的网络(对数因子)。我们评估所提出的算法的有效性上一个强大的逻辑斯谛回归问题。

英文摘要:

We study solution methods for (strongly-)convex-(strongly)-concave Saddle-Point Problems (SPPs) over networks of two type - master/workers (thus centralized) architectures and meshed (thus decentralized) networks. The local functions at each node are assumed to be similar, due to statistical data similarity or otherwise. We establish lower complexity bounds for a fairly general class of algorithms solving the SPP. We show that a given suboptimality $ε>0$ is achieved over master/workers networks in $Ω\big(Δ\cdot δ/μ\cdot \log (1/\varepsilon)\big)$ rounds of communications, where $δ>0$ measures the degree of similarity of the local functions, $μ$ is their strong convexity constant, and $Δ$ is the diameter of the network. The lower communication complexity bound over meshed networks reads $Ω\big(1/{\sqrtρ} \cdot δ/μ\cdot\log (1/\varepsilon)\big)$, where $ρ$ is the (normalized) eigengap of the gossip matrix used for the communication between neighbouring nodes. We then propose algorithms matching the lower bounds over either types of networks (up to log-factors). We assess the effectiveness of the proposed algorithms on a robust logistic regression problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑