Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the sense that even simply verifying that a given point is a local minimum can be NP-hard [1]. Still, some relatively simple algorithms have been shown to lead to surprisingly good empirical results in many contexts of interest. Perhaps the most prominent example is the success of the backpropagation algorithm for training neural networks. Several recent works have pursued rigorous analytical justification for this phenomenon by studying the structure of the nonconvex optimization problems and establishing that simple algorithms, such as gradient descent and its variations, perform well in converging towards local minima and avoiding saddle-points. A key insight in these analyses is that gradient perturbations play a critical role in allowing local descent algorithms to efficiently distinguish desirable from undesirable stationary points and escape from the latter. In this article, we cover recent results on second-order guarantees for stochastic first-order optimization algorithms in centralized, federated, and decentralized architectures.
University of Science and Technology of China（中国科学技术大学）；Infinia ML, Inc.（Infinia ML公司）
Decentralized optimization algorithms have attracted intensive interests recently, as it has a balanced communication pattern, especially when solving large-scale machine learning problems. Stochastic Path Integrated Differential Estimator Stochastic First-Order method (SPIDER-SFO) nearly achieves the algorithmic lower bound in certain regimes for nonconvex problems. However, whether we can find a decentralized algorithm which achieves a similar convergence rate to SPIDER-SFO is still unclear. To tackle this problem, we propose a decentralized variant of SPIDER-SFO, called decentralized SPIDER…
University of Rochester（罗切斯特大学）；ETH Zurich（苏黎世联邦理工学院）；University of California, Davis（加利福尼亚大学戴维斯分校）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Tencent AI lab（腾讯人工智能实验室）
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are built in a centralized fashion. One bottleneck of centralized algorithms lies on high communication cost on the central node. Motivated by this, we ask, can decentralized algorithms be faster than its centralized counterpart? Although decentralized PSGD (D-PSGD) algorithms have been studied by the control community, existing analysis and theory do not show any advantage over centralized PSGD (C-PSGD) algorithms, simply assuming the application scenario where only the decentralized network is available. In th…
The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold). Although many optimization algorithms for minimax problems have been developed in the Euclidean setting, it is difficult to convert them into Riemannian cases, and algorithms for nonconvex minimax problems with nonconvex constraints are even rare. On the other hand, to address the big data challenges, decentralized (serverless) training techniques have re…
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization
SGIT AI Lab, State Grid Corporation of China（国网公司SGIT AI实验室）；School of Data Science, Fudan University（复旦大学数据科学学院）；Shanghai Key Laboratory for Contemporary Applied Mathematics（上海市现代应用数学重点实验室）
Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
Department of Mathematical Sciences（数学科学系）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Department of Computer Science（计算机科学系）；MIT-IBM Watson AI Lab, IBM Research（MIT-IBM Watson AI实验室，IBM研究）；IBM T. J. Watson Research Center（IBM T.J. 华生研究中心）
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（工业和信息化部模式分析与机器智能重点实验室）
We design a low complexity decentralized learning algorithm to train a recently proposed large neural network in distributed processing nodes (workers). We assume the communication network between the workers is synchronized and can be modeled as a doubly-stochastic mixing matrix without having any master node. In our setup, the training data is distributed among the workers but is not shared in the training process due to privacy and security concerns. Using alternating-direction-method-of-multipliers (ADMM) along with a layerwise convex optimization approach, we propose a decentralized learn…
University of Rochester（罗切斯特大学）；University of Michigan（密歇根大学）；Kwai Inc（快手公司）；ETH Zurich（苏黎世联邦理工学院）；Hong Kong University of Science and Technology（香港科技大学）
Communication is a key bottleneck in distributed training. Recently, an \emph{error-compensated} compression technology was particularly designed for the \emph{centralized} learning and receives huge successes, by showing significant advantages over state-of-the-art compression based methods in saving the communication cost. Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized lea…
NTT DATA Mathematical Systems Inc.（NTT数据数学系统公司）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）
In recent centralized nonconvex distributed learning and federated learning, local methods are one of the promising approaches to reduce communication time. However, existing work has mainly focused on studying first-order optimality guarantees. On the other side, second-order optimality guaranteed algorithms, i.e., algorithms escaping saddle points, have been extensively studied in the non-distributed optimization literature. In this paper, we study a new local algorithm called Bias-Variance Reduced Local Perturbed SGD (BVR-L-PSGD), that combines the existing bias-variance reduced gradient es…
Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
State Key Laboratory of Digital Intelligent Modeling and Simulation（数字智能建模与仿真国家重点实验室）；National University of Defense Technology（国防科技大学）；School of Cyber Science and Technology（网络安全科学与技术学院）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）
Consensus optimization has received considerable attention in recent years. A number of decentralized algorithms have been proposed for {convex} consensus optimization. However, to the behaviors or consensus \emph{nonconvex} optimization, our understanding is more limited. When we lose convexity, we cannot hope our algorithms always return global solutions though they sometimes still do sometimes. Somewhat surprisingly, the decentralized consensus algorithms, DGD and Prox-DGD, retain most other properties that are known in the convex setting. In particular, when diminishing (or constant) step …
Peking University（北京大学）；Tencent AI Lab（腾讯人工智能实验室）；BAAI（北京智源人工智能研究院）
In cooperative multi-agent reinforcement learning (MARL), combining value decomposition with actor-critic enables agents to learn stochastic policies, which are more suitable for the partially observable environment. Given the goal of learning local policies that enable decentralized execution, agents are commonly assumed to be independent of each other, even in centralized training. However, such an assumption may prohibit agents from learning the optimal joint policy. To address this problem, we explicitly take the dependency among agents into centralized training. Although this leads to the…
Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?
Zhejiang University（浙江大学）；Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance, School of Computer and Computing Science, Hangzhou City University（浙江省实时智慧城市治理实时智能技术工程研究中心，计算机与计算科学学院，杭州市大学）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江区）区块链与数据安全研究院）
University of Science and Technology of China（中国科学技术大学）；Huawei Cloud（华为云）
Due to the partial observability and communication constraints in many multi-agent reinforcement learning (MARL) tasks, centralized training with decentralized execution (CTDE) has become one of the most widely used MARL paradigms. In CTDE, centralized information is dedicated to learning the allocation of the team reward with a mixing network, while the learning of individual Q-values is usually based on local observations. The insufficient utility of global observation will degrade performance in challenging environments. To this end, this work proposes a novel Centralized Teacher with Decen…
University of Delaware（特拉华大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
Recent multi-agent actor-critic methods have utilized centralized training with decentralized execution to address the non-stationarity of co-adapting agents. This training paradigm constrains learning to the centralized phase such that only pre-learned policies may be used during the decentralized phase, which performs poorly when agent communications are delayed, noisy, or disrupted. In this work, we propose a new system that can gracefully handle partially-observable information due to communication disruptions during decentralized execution. Our approach augments the multi-agent actor-crit…
Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models
Tsinghua University（清华大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究所（TeleAI），中国电信）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiaotong University（上海交通大学）
Degrees of Freedom in Multi-Agent Coordination: Between Centralized and Decentralized Regimes
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Wake Forest University（维克森林大学）；Shandong University（山东大学）；University of Science and Technology of China（中国科学技术大学）
The infectious diseases are spreading due to human interactions enabled by various social networks. Therefore, when a new pathogen such as SARS-CoV-2 causes an outbreak, the non-pharmaceutical isolation strategies (e.g., social distancing) are the only possible response to disrupt its spreading. To this end, we introduce the new epidemic model (SICARS) and compare the centralized (C), decentralized (D), and combined (C+D) social distancing strategies, and analyze their efficiency to control the dynamics of COVID-19 on heterogeneous complex networks. Our analysis shows that the centralized soci…
Public and private blockchain for decentralized digital building twins and building automation system
School of Technology, Eastern Illinois University（东部伊利诺伊大学技术学院）；Myers-Lawson School of Construction, Virginia Polytechnic Institute and State University（弗吉尼亚理工学院和州立大学梅斯-劳森建设学院）
Robotic systems, working together as a team, are becoming valuable players in different real-world applications, from disaster response to warehouse fulfillment services. Centralized solutions for coordinating multi-robot teams often suffer from poor scalability and vulnerability to communication disruptions. This paper develops a decentralized multi-agent task allocation (Dec-MATA) algorithm for multi-robot applications. The task planning problem is posed as a maximum-weighted matching of a bipartite graph, the solution of which using the blossom algorithm allows each robot to autonomously id…
Large scale power systems are comprised of regional utilities with assets that stream sensor readings in real time. In order to detect cyberattacks, the globally acquired, real time sensor data needs to be analyzed in a centralized fashion. However, owing to operational constraints, such a centralized sharing mechanism turns out to be a major obstacle. In this paper, we propose a blockchain based decentralized framework for detecting coordinated replay attacks with full privacy of sensor data. We develop a Bayesian inference mechanism employing locally reported attack probabilities that is tai…
This paper presents an algorithm for a team of mobile robots to simultaneously learn a spatial field over a domain and spatially distribute themselves to optimally cover it. Drawing from previous approaches that estimate the spatial field through a centralized Gaussian process, this work leverages the spatial structure of the coverage problem and presents a decentralized strategy where samples are aggregated locally by establishing communications through the boundaries of a Voronoi partition. We present an algorithm whereby each robot runs a local Gaussian process calculated from its own measu…
