In this paper, we consider the problem of distributed consensus optimization over multi-agent networks with directed network topology. Assuming each agent has a local cost function that is smooth and strongly convex, the global objective is to minimize the average of all the local cost functions. To solve the problem, we introduce a robust gradient tracking method (R-Push-Pull) adapted from the recently proposed Push-Pull/AB algorithm. R-Push-Pull inherits the advantages of Push-Pull and enjoys linear convergence to the optimal solution with exact communication. Under noisy information exchange, R-Push-Pull is more robust than the existing gradient tracking based algorithms; the solutions obtained by each agent reach a neighborhood of the optimum in expectation exponentially fast under a constant stepsize policy. We provide a numerical example that demonstrate the effectiveness of R-Push-Pull.
Communication efficiency is a major bottleneck in the applications of distributed networks. To address the problem, the problem of quantized distributed optimization has attracted a lot of attention. However, most of the existing quantized distributed optimization algorithms can only converge sublinearly. To achieve linear convergence, this paper proposes a novel quantized distributed gradient tracking algorithm (Q-DGT) to minimize a finite sum of local objective functions over directed networks. Moreover, we explicitly derive the update rule for the number of quantization levels, and prove th…
This work builds on our recent work on a distributed optimization algorithm for graphs with directed unreliable communications. We show its linear convergence when we take either the proximal of each function or an affine minorant for when the function is smooth.
Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks
Shanghai Center for Mathematical Sciences, Fudan University（复旦大学上海数学中心）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；Department of Mathematics, Michigan State University（密歇根州立大学数学系）
This paper studies cooperative tracking problem of heterogeneous Euler-Lagrange systems with an uncertain leader. Different from most existing works, system dynamic knowledge of the leader node is unaccessible to any follower node in our paper. Distributed adaptive observers are designed for all follower nodes, simultaneously estimate the state and parameters of the leader node. The observer design does not rely on the frequency knowledge of the leader node, and the estimation errors are shown to converge to zero exponentially. Moreover, the results are applied to general directed graphs, wher…
We show that the averaged consensus algorithm on directed graphs with unreliable communications by Bof-Carli-Schenato has a dual optimization interpretation, which could be extended to the case of distributed optimization. We report on our numerical simulations for the distributed optimization algorithm for smooth and nonsmooth functions.
We propose a distributed algorithm, termed the Directed-Distributed Projected Subgradient (D-DPS), to solve a constrained optimization problem over a multi-agent network, where the goal of agents is to collectively minimize the sum of locally known convex functions. Each agent in the network owns only its local objective function, constrained to a commonly known convex set. We focus on the circumstance when communications between agents are described by a directed network. The D-DPS augments an additional variable for each agent, to overcome the asymmetry caused by the directed communication n…
Motivated by emerging applications in wireless sensor networks and large-scale data processing, we consider distributed optimization over directed networks where the agents communicate their information locally to their neighbors to cooperatively minimize a global cost function. We introduce a new unifying distributed constrained optimization model that is characterized as a bilevel optimization problem. This model captures a wide range of existing problems over directed networks including: (i) Distributed optimization with linear constraints; (ii) Distributed unconstrained nonstrongly convex …
Sporadic Gradient Tracking over Directed Graphs: A Theoretical Perspective on Decentralized Federated Learning
School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA（电气与计算机工程学院，普渡大学，西拉法基，印第安纳州，美国）；Department of Computer Science and Engineering, Yonsei University, Seoul, South Korea（计算机科学与工程系，延世大学，首尔，韩国）；Department of Electrical Engineering, University at Buffalo -- SUNY, Buffalo, NY, USA（电气工程系，布法罗大学——SUNY，布法罗，纽约州，美国）
In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method. We show that, in expectation, the iterates generated by each agent are attracted to a neighborhood of the optimal solution, where they accumulate exponential…
This paper considers the cooperative output regulation problem for linear multi-agent systems with a directed communication graph, heterogeneous linear subsystems, and an exosystem whose output is available to only a subset of subsystems. Both the cases with nominal and uncertain linear subsystems are studied. For the case with nominal linear subsystems, a distributed adaptive observer-based controller is designed, where the distributed adaptive observer is implemented for the subsystems to estimate the exogenous signal. For the case with uncertain linear subsystems, the proposed distributed o…
This paper considers a distributed convex optimization problem over a time-varying multi-agent network, where each agent has its own decision variables that should be set so as to minimize its individual objective subject to local constraints and global coupling equality constraints. Over directed graphs, a distributed algorithm is proposed that incorporates the push-sum protocol into dual subgradient methods. Under the convexity assumption, the optimality of primal and dual variables, and constraint violations is first established. Then the explicit convergence rates of the proposed algorithm…
In this paper, we consider distributed optimization problems where the goal is to minimize a sum of objective functions over a multi-agent network. We focus on the case when the inter-agent communication is described by a strongly-connected, \emph{directed} graph. The proposed algorithm, ADD-OPT (Accelerated Distributed Directed Optimization), achieves the best known convergence rate for this class of problems,~$O(μ^{k}),0<μ<1$, given strongly-convex, objective functions with globally Lipschitz-continuous gradients, where~$k$ is the number of iterations. Moreover, ADD-OPT supports a wider and …
We consider a multi agent optimization problem where a set of agents collectively solves a global optimization problem with the objective function given by the sum of locally known convex functions. We focus on the case when information exchange among agents takes place over a directed network and propose a distributed subgradient algorithm in which each agent performs local processing based on information obtained from his incoming neighbors. Our algorithm uses weight balancing to overcome the asymmetries caused by the directed communication network, i.e., agents scale their outgoing informat…
North Carolina State University（北卡罗来纳州立大学）；Oklahoma State University（俄克拉荷马州立大学）；DEVCOM U.S. Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）
Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local c…
This paper aims at addressing distributed averaging problems for signed networks in the presence of general directed topologies that are represented by signed digraphs. A new class of improved Laplacian potential functions is proposed by presenting two notions of any signed digraph: induced unsigned digraph and mirror (undirected) signed graph, based on which two distributed averaging protocols are designed using the nearest neighbor rules. It is shown that with any of the designed protocols, signed-average consensus (respectively, state stability) can be achieved if and only if the associated…
In this paper, we propose a distributed algorithm, called Directed-Distributed Gradient Descent (D-DGD), to solve multi-agent optimization problems over directed graphs. Existing algorithms mostly deal with similar problems under the assumption of undirected networks, i.e., requiring the weight matrices to be doubly-stochastic. The row-stochasticity of the weight matrix guarantees that all agents reach consensus, while the column-stochasticity ensures that each agent's local gradient contributes equally to the global objective. In a directed graph, however, it may not be possible to construct …
We study distributed big-data nonconvex optimization in multi-agent networks. We consider the (constrained) minimization of the sum of a smooth (possibly) nonconvex function, i.e., the agents' sum-utility, plus a convex (possibly) nonsmooth regularizer. Our interest is on big-data problems in which there is a large number of variables to optimize. If treated by means of standard distributed optimization algorithms, these large-scale problems may be intractable due to the prohibitive local computation and communication burden at each node. We propose a novel distributed solution method where, a…
We study the problem of constrained distributed optimization in multi-agent networks when some of the computing agents may be faulty. In this problem, the system goal is to have all the non-faulty agents collectively minimize a global objective given by weighted average of local cost functions, each of which is initially known to a non-faulty agent only. In particular, we are interested in the scenario when the computing agents are connected by an arbitrary directed communication network, some of the agents may suffer from crash faults or Byzantine faults, and the estimate of each agent is res…
We propose Directed-Distributed Projected Subgradient (D-DPS) to solve a constrained optimization problem over a multi-agent network, where the goal of agents is to collectively minimize the sum of locally known convex functions. Each agent in the network owns only its local objective function, constrained to a commonly known convex set. We focus on the circumstance when communications between agents are described by a \emph{directed} network. The D-DPS combines surplus consensus to overcome the asymmetry caused by the directed communication network. The analysis shows the convergence rate to …
We study the distributed stochastic compositional optimization problems over directed communication networks in which agents privately own a stochastic compositional objective function and collaborate to minimize the sum of all objective functions. We propose a distributed stochastic compositional gradient descent method, where the gradient tracking and the stochastic correction techniques are employed to adapt to the networks' directed structure and increase the accuracy of inner function estimation. When the objective function is smooth, the proposed method achieves the convergence rate $\ma…
This paper develops a fast distributed algorithm, termed \emph{DEXTRA}, to solve the optimization problem when~$n$ agents reach agreement and collaboratively minimize the sum of their local objective functions over the network, where the communication between the agents is described by a~\emph{directed} graph. Existing algorithms solve the problem restricted to directed graphs with convergence rates of $O(\ln k/\sqrt{k})$ for general convex objective functions and $O(\ln k/k)$ when the objective functions are strongly-convex, where~$k$ is the number of iterations. We show that, with the approp…
This paper investigates the distributed continuous-time nonconvex optimization problem over unbalanced directed networks. The objective is to cooperatively drive all the agent states to an optimal solution that minimizes the sum of the local cost functions. Based on the topology balancing technique and adaptive control approach, a novel fully distributed algorithm is developed for each agent with neither prior global information concerning network connectivity nor convexity of local cost functions. By viewing the proposed algorithm as a perturbed system, its input-to-state stability with a van…
FAB: A First-Order AB-based Gradient Algorithm for Distributed Bilevel Optimization over Time-Varying Directed Graphs
Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；School of Artificial Intelligence, Southern University of Science and Technology（南方科技大学人工智能学院）；National Center for Applied Mathematics Shenzhen（深圳应用数学国家研究中心）；Detection Institute for Advanced Technology Longhua-Shenzhen（深圳龙华先进技术检测研究院）
This paper presents a novel distributed algorithm for tracking a maneuvering target using bearing or direction of arrival measurements collected by a networked sensor array. The proposed approach is built on the dynamic average-consensus algorithm, which allows a networked group of agents (nodes) to reach consensus on the global average of a set of local time-varying signals in a distributed fashion. Since the average-consensus error corresponding to the presented dynamic average-consensus algorithm converges to zero in finite time, the proposed distributed algorithm guarantees that the tracki…
This paper explores distributed Nash equilibrium seeking problems for games in which the players have limited knowledge on other players' actions. In particular, the involved players are considered to be high-order integrators with their control inputs constrained within a pre-specified region. A linear transformation for the players' dynamics is firstly utilized to facilitate the design of bounded control inputs incorporating multiple saturation functions. By introducing consensus protocols with adaptive and time-varying gains, the unknown actions for players are distributively estimated. The…
In this paper, we aim to develop distributed continuous-time algorithms over directed graphs to seek the Nash equilibrium in a noncooperative game. Motivated by the recent consensus-based designs, we present a distributed algorithm with a proportional gain for weight-balanced directed graphs. By further embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the graph Laplacian, we extend it to the case with arbitrary strongly connected directed graphs having possible unbalanced weights. In both cases, the Nash equilibrium is proven to be exactly reached wi…
In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method (DSGT) and a gossip-like stochastic gradient tracking method (GSGT). We show that, in expectation, the iterates generated by each agent are attracted to a nei…
In this paper, we propose a framework under which the decentralized optimization algorithms suggested in \cite{JKJJ,MA, NO,NO2} can be treated in a unified manner. More precisely, we show that the distributed subgradient descent algorithms \cite{JKJJ, NO}, the subgradient-push algorithm \cite{NO2}, and the distributed algorithm with row-stochastic matrix \cite{MA} can be derived by making suitable choices of consensus matrices, step-size and subgradient from the decentralized subgradient descent proposed in \cite{NO}. As a result of such unified understanding, we provide a convergence proof th…
Semnan University（塞姆南大学）；Aalto University（阿尔托大学）；Georgia State University（佐治亚州立大学）；Tufts University（塔夫茨大学）
In this paper, we consider the binary classification problem via distributed Support-Vector-Machines (SVM), where the idea is to train a network of agents, with limited share of data, to cooperatively learn the SVM classifier for the global database. Agents only share processed information regarding the classifier parameters and the gradient of the local loss functions instead of their raw data. In contrast to the existing work, we propose a continuous-time algorithm that incorporates network topology changes in discrete jumps. This hybrid nature allows us to remove chattering that arises beca…
This paper addresses distributed average tracking for a group of heterogeneous physical agents consisting of single-integrator, double-integrator and Euler-Lagrange dynamics. Here, the goal is that each agent uses local information and local interaction to calculate the average of individual time-varying reference inputs, one per agent. Two nonsmooth algorithms are proposed to achieve the distributed average tracking goal. In our first proposed algorithm, each agent tracks the average of the reference inputs, where each agent is required to have access to only its own position and the relative…
In this work, we study the classical distributed optimization problem over digraphs, where the objective function is a sum of smooth local functions. Inspired by the implicit tracking mechanism proposed in our earlier work, we develop a unified algorithmic framework from a pure primal perspective, i.e., UGT, which is essentially a generalized gradient tracking method and can unify most existing distributed optimization algorithms with constant step-sizes. It is proved that two variants of UGT can both achieve linear convergence if the global objective function is strongly convex. Finally, the …
We consider the distributed optimization problem, the goal of which is to minimize the sum of local objective functions over a directed network. Though it has been widely studied recently, most of the existing algorithms are designed for synchronized or randomly activated implementation, which may create deadlocks in practice. In sharp contrast, we propose a \emph{fully} asynchronous push-pull gradient algorithm (APPG) where each node updates without waiting for any other node by using (possibly stale) information from neighbors. Thus, it is both deadlock-free and robust to any bounded communi…
A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization
Engineering Research Center of Metallurgical Automation and Measurement Technology（冶金自动化与测量技术工程研究中心）；Wuhan University of Science and Technology（武汉科技大学）；College of Control Science and Engineering（控制科学与工程学院）；Zhejiang University（浙江大学）
We study the asymptotic properties of distributed consensus algorithms over switching directed random networks. More specifically, we focus on consensus algorithms over independent and identically distributed, directed random graphs, where each agent can communicate with any other agent with some exogenously specified probability. While different aspects of consensus algorithms over random switching networks have been widely studied, a complete characterization of the distribution of the asymptotic value for general \textit{asymmetric} random consensus algorithms remains an open problem. In th…
In this paper, two distributed multi-proximal primal-dual algorithms are proposed to deal with a class of distributed nonsmooth resource allocation problems. In these problems, the global cost function is the summation of local convex and nonsmooth cost functions, each of which consists of one twice differentiable function and multiple nonsmooth functions. Communication graphs of underling multi-agent systems are directed and strongly connected but not necessarily weighted-balanced. The multi-proximal splitting is designed to deal with the difficulty caused by the unproximable property of the …
Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networks
School of Electrical, Computer and Energy Engineering, Arizona State University（亚利桑那州立大学电气、计算机与能源工程学院）；Department of Electrical and Computer Engineering, Princeton University（普林斯顿大学电气与计算机工程系）
In this paper, we study the problem of consensus-based distributed optimization where a network of agents, abstracted as a directed graph, aims to minimize the sum of all agents' cost functions collaboratively. In existing distributed optimization approaches (Push-Pull/AB) for directed graphs, all agents exchange their states with neighbors to achieve the optimal solution with a constant stepsize, which may lead to the disclosure of sensitive and private information. For privacy preservation, we propose a novel state-decomposition based gradient tracking approach (SD-Push-Pull) for distributed…
In this paper we consider a recently developed distributed optimization algorithm based on gradient tracking. We propose a system theory framework to analyze its structural properties on a preliminary, quadratic optimization set-up. Specifically, we focus on a scenario in which agents in a static network want to cooperatively minimize the sum of quadratic cost functions. We show that the gradient tracking distributed algorithm for the investigated program can be viewed as a sparse closed-loop linear system in which the dynamic state-feedback controller includes consensus matrices and optimizat…
In this paper, we study consensus problem in multi-agent system with directed topology by event-triggered feedback control. That is, at each agent, the diffusion coupling feedbacks are based on the information from its latest observations to its in-neighbours. We derive distributed criteria to determine the next observation time of each agent that are triggered by its in-neighbours' information and its own states respectively. We prove that if the network topology is irreducible, then under the event-triggered coupling principles, the multi-agent system reach consensus. Then, we extend these r…
This paper focuses on a time-varying Nash equilibrium trajectory tracking problem, that is applicable to a wide range of non-cooperative game applications arising in dynamic environments. To solve this problem, we propose a distributed prediction correction algorithm, termed DPCA, in which each player predicts future strategies based on previous observations and then exploits predictions to effectively track the NE trajectory by using one or multiple distributed gradient descent steps across a network. We rigorously demonstrate that the tracking sequence produced by the proposed algorithm is a…
We investigate a distributed optimization problem over a cooperative multi-agent time-varying network, where each agent has its own decision variables that should be set so as to minimize its individual objective subject to local constraints and global coupling constraints. Based on push-sum protocol and dual decomposition, we design a distributed regularized dual gradient algorithm to solve this problem, in which the algorithm is implemented in time-varying directed graphs only requiring the column stochasticity of communication matrices. By augmenting the corresponding Lagrangian function wi…
This paper studies a class of distributed optimization algorithms by a set of agents, where each agent has only access to its own local convex objective function, and jointly minimizes the sum of the functions. The communications among agents are described by a sequence of time-varying directed graphs which are assumed to be uniformly strongly connected. A column stochastic mixing matrices is employed in the algorithm, which also exactly steers all the agents to asymptotically converge to a global and consensual optimal solution even under the assumption that the step-sizes are uncoordinated. …
