arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

通过自适应正则化在节点故障下进行鲁棒分散优化

Robust Decentralized Optimization under Node Failures via Adaptive Regularization

Ilya Kuruzov, Anastasiia Murzina

arXiv 2607.09939首次发表:更新:

AI 中文总结

研究网络中节点故障下的分散优化问题,提出遗留梯度跟踪方法,证明其能使误差界几何衰减,而经典方法有永不衰减的偏差,数值实验验证了理论。

AI 中文摘要

我们研究在网络中对函数和进行分散最小化,其中节点可能离开,其余节点保持连接,且拓扑在离开之间冻结。标准方法会遗忘已离开节点的函数,导致与数据异质性成比例的永久偏差。我们提出了遗留梯度跟踪(Legacy - GT):在离开前,节点将其函数压缩为梯度锚定二次遗留,遗赠给邻居并提供精确保持梯度跟踪不变量的校正。我们证明了最优遗留曲率是强凸性和平滑性常数的平均值,遗留可在链式离开中无损组合,自适应锚定规则产生的误差界在最后一次离开后几何衰减到一个小残差——离开时网络优化误差及其异质性半径的最小值。相比之下,经典的丢弃并遗忘基线存在永不衰减的偏差。数值实验证实了该理论。

英文摘要

We study decentralized minimization of a sum of functions over a network where nodes may only leave, the remaining nodes stay connected, and the topology freezes between departures. Standard methods forget departed functions, causing a permanent bias proportional to the heterogeneity of the data. We propose Legacy Gradient Tracking (Legacy-GT): before leaving, a node compresses its function into a gradient-anchored quadratic legacy, bequeaths it to a neighbor, and provides a correction that exactly preserves the gradient-tracking invariant. We prove that the optimal legacy curvature is the average of the strong-convexity and smoothness constants, that legacies compose losslessly across chained departures, and that an adaptive anchor rule yields error bounds that decay geometrically after the last departure to a small residual--the minimum of the network's optimization error at departure and its heterogeneity radius. In contrast, the classic drop-and-forget baseline suffers a bias that never decays. Numerical experiments confirm the theory.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑