The recent deployment of multi-agent networks has enabled the distributed solution of learning problems, where agents cooperate to train a global model without sharing their local, private data. This work specifically targets some prevalent challenges inherent to distributed learning: (i) online training, i.e., the local data change over time; (ii) asynchronous agent computations; (iii) unreliable and limited communications; and (iv) inexact local computations. To tackle these challenges, we apply the Distributed Operator Theoretical (DOT) version of the Alternating Direction Method of Multipliers (ADMM), which we call "DOT-ADMM". We prove that if the DOT-ADMM operator is metric subregular, then it converges with a linear rate for a large class of (not necessarily strongly) convex learning problems toward a bounded neighborhood of the optimal time-varying solution, and characterize how such neighborhood depends on (i)-(iv). We first derive an easy-to-verify condition for ensuring the metric subregularity of an operator, followed by tutorial examples on linear and logistic regression problems. We corroborate the theoretical analysis with numerical simulations comparing DOT-ADMM with other state-of-the-art algorithms, showing that only the proposed algorithm exhibits robustness to (i)-(iv).
LEARN: An Invex Loss for Outlier Oblivious Robust Online Optimization
Institute of Data Science（数据科学研究所）；National University of Singapore（新加坡国立大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Mathematics（数学系）
Online Robust Subspace Tracking from Partial Information
Nanjing University of Information Science and Technology（南京信息工程大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Chinese University of Hong Kong（香港中文大学）
Hybrid Quantum-Classical Algorithm For Robust Optimization via Stochastic-Gradient Online Learning
Centre for Quantum Technologies, National University of Singapore（新加坡国立大学量子技术中心）；Center for Quantum Computing Science, University of Latvia（拉脱维亚大学量子计算科学中心）；HUN-REN Alfréd Rényi Institute of Mathematics（匈牙利研究与教育网络阿尔弗雷德·雷尼数学研究所）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）
