在这篇文章中,我们提出了一种新的方法,优化然后同意最小化的总和$ f = \sum_{i=1}^n f_i(x)$凸目标函数在有向图。优化然后同意方法在分布式优化框架中将优化步骤和共识步骤合并。优化然后同意的关键动机是保证在分布式优化算法的每次迭代期间代理的估计之间的分歧保持在任何先验指定的公差下;现有算法不提供在许多实际场景中所需的这样的保证。在这种方法中,每个代理在每次迭代期间保持对最优解的估计,并利用其局部可用的梯度信息以及有限时间近似共识协议来向最优解移动(因此称为一致性共识算法)。我们建立了所提出的算法具有全局R-线性收敛速度,如果聚合函数$f$是强凸和Lipschitz可微。在放松的$f_i$是凸的和Lipschitz可微的假设下,我们还证明了目标函数误差残差以Q-线性速率减小(就梯度计算步骤的数量而言),直到它达到一个小值,这可以使用有限时间近似共识协议上指定的容差值来管理;当$f_i$不一定是强凸函数时,文献中没有现有的方法具有如此强的收敛保证。与传统算法的$O(k)$相比,改进的算法在满足约束条件和更好收敛性的保证方面的通信开销是$O(k\log k)$迭代次数。此外,我们数值评估所提出的算法的性能,通过解决分布式逻辑回归问题。
英文摘要:
In this article, we propose a new approach, optimize then agree for minimizing a sum $ f = \sum_{i=1}^n f_i(x)$ of convex objective functions over a directed graph. The optimize then agree approach decouples the optimization step and the consensus step in a distributed optimization framework. The key motivation for optimize then agree is to guarantee that the disagreement between the estimates of the agents during every iteration of the distributed optimization algorithm remains under any apriori specified tolerance; existing algorithms do not provide such a guarantee which is required in many practical scenarios. In this method, each agent during each iteration maintains an estimate of the optimal solution and, utilizes its locally available gradient information along with a finite-time approximate consensus protocol to move towards the optimal solution (hence the name Gradient-Consensus algorithm). We establish that the proposed algorithm has a global R-linear rate of convergence if the aggregate function $f$ is strongly convex and Lipschitz differentiable. We also show that under the relaxed assumption of $f_i$'s being convex and Lipschitz differentiable, the objective function error residual decreases at a Q-linear rate (in terms of the number of gradient computation steps) until it reaches a small value, which can be managed using the tolerance value specified on the finite-time approximate consensus protocol; no existing method in the literature has such strong convergence guarantees when $f_i$ are not necessarily strongly convex functions. The communication overhead for the improved guarantees on meeting constraints and better convergence of our algorithm is $O(k\log k)$ iterates in comparison to $O(k)$ of the traditional algorithms. Further, we numerically evaluate the performance of the proposed algorithm by solving a distributed logistic regression problem.
Designing Distributed Fixed-Time Consensus Protocols for Linear Multi-Agent Systems Over Directed Graphs
School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);Department of Electronic Engineering, City University of Hong Kong(香港城市大学电子工程系)
Event-triggered consensus of multi-agent systems under directed topology based on periodic sampled-data
School of Mathematics and Statistics, Qingdao University(青岛大学数学与统计学院);College of Automation and Electrical Engineering, Qingdao University(青岛大学自动化与电气工程学院);School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院)
Fully distributed consensus control for stochastic multi-agent systems under undirected and directed topologies
China University of Geosciences(中国地质大学);Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems(复杂系统先进控制与智能自动化湖北省重点实验室)
Non-trivial consensus on directed signed matrix-weighted networks with compound measurement noises and time-varying topologies
School of Mathematics and Statistics, Wuhan University(武汉大学数学与统计学院);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院)
Decentralized design of consensus protocols with minimal communication links based on directed spanning tree
College of Artificial Intelligence and Automation, Beijing University of Technology(北京工业大学人工智能与自动化学院);Engineering Research Center of Digital Community, Ministry of Education(教育部数字社区工程研究中心)
Guaranteed Cost Tracking for Uncertain Coupled Multi-agent Systems Using Consensus over a Directed Graph
The University of New South Wales at the Australian Defence Force Academy(澳大利亚国防军学院新南威尔士大学);Australian National University(澳大利亚国立大学);Southeast University(东南大学)
Max-Consensus with Deterministic Convergence in Directed Graphs with Unreliable Communication Links
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);University of Cyprus(塞浦路斯大学);Aalto University(阿尔托大学);KTH Royal Institute of Technology(瑞典皇家理工学院)
Grounded Laplacians of Directed Signed Matrix-Weighted Networks: Spectral Properties and Applications to Non-Trivial Consensus
the College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);the School of Mathematics and Statistics, Wuhan University(武汉大学数学与统计学院);the Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology(深圳先进技术大学计算机与控制工程学院)
Reaching a Consensus in Networks of High-Order Integral Agents under Switching Directed Topology
State Key Laboratory of Intelligent Control and Management of Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统智能控制与管理国家重点实验室)
Almost Decouplability of any Directed Weighted Network Topology
College of Electrical Engineering, Northwest University for Nationalities(西北民族大学电气工程学院);PN Engineering College, National University of Sciences and Technology(国立科技大学工程与科学学院)
Global $H_\infty$ Consensus of Multi-Agent Systems with Lipschitz Nonlinear Dynamics
School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院)
Institut de Matemàtica Multidisciplinària, Universitat Politècnica de València(多学科数学研究所,瓦伦西亚理工大学);Departament de Sistemes Informàtics i Computació, Universitat Politècnica de València(信息系统与计算系,瓦伦西亚理工大学)