This paper studies a system of linear equations, denoted as $Ax = b$, which is horizontally partitioned (rows in $A$ and $b$) and stored over a network of $m$ devices connected in a fixed directed graph. We design a fast distributed algorithm for solving such a partitioned system of linear equations, that additionally, protects the privacy of local data against an honest-but-curious adversary that corrupts at most $τ$ nodes in the network. First, we present TITAN, privaTe fInite Time Average coNsensus algorithm, for solving a general average consensus problem over directed graphs, while protecting statistical privacy of private local data against an honest-but-curious adversary. Second, we propose a distributed linear system solver that involves each agent/devices computing an update based on local private data, followed by private aggregation using TITAN. Finally, we show convergence of our solver to the least squares solution in finite rounds along with statistical privacy of local linear equations against an honest-but-curious adversary provided the graph has weak vertex-connectivity of at least $τ+1$. We perform numerical experiments to validate our claims and compare our solution to the state-of-the-art methods by comparing computation, communication and memory costs.
In this paper, we study the problem of consensus-based distributed optimization where a network of agents, abstracted as a directed graph, aims to minimize the sum of all agents' cost functions collaboratively. In existing distributed optimization approaches (Push-Pull/AB) for directed graphs, all agents exchange their states with neighbors to achieve the optimal solution with a constant stepsize, which may lead to the disclosure of sensitive and private information. For privacy preservation, we propose a novel state-decomposition based gradient tracking approach (SD-Push-Pull) for distributed…
Distributed algorithms enable private Optimal Power Flow (OPF) computations by avoiding the need in sharing sensitive information localized in algorithms sub-problems. However, adversaries can still infer this information from the coordination signals exchanged across iterations. This paper seeks formal privacy guarantees for distributed OPF computations and provides differentially private algorithms for OPF computations based on the consensus Alternating Direction Method of Multipliers (ADMM). The proposed algorithms attain differential privacy by introducing static and dynamic random perturb…
We study a class of distributed convex constrained optimization problems where a group of agents aim to minimize the sum of individual objective functions while each desires that any information about its objective function is kept private. We prove the impossibility of achieving differential privacy using strategies based on perturbing the inter-agent messages with noise when the underlying noise-free dynamics are asymptotically stable. This justifies our algorithmic solution based on the perturbation of individual functions with Laplace noise. To this end, we establish a general framework fo…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Continual data collection and widespread deployment of machine learning algorithms, particularly the distributed variants, have raised new privacy challenges. In a distributed machine learning scenario, the dataset is stored among several machines and they solve a distributed optimization problem to collectively learn the underlying model. We present a secure multi-party computation inspired privacy preserving distributed algorithm for optimizing a convex function consisting of several possibly non-convex functions. Each individual objective function is privately stored with an agent while the…
UCLA Electrical and Computer Engineering Department（加州大学洛杉矶分校电气与计算机工程系）；Amazon Web Services（亚马逊云科技）
Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under data privacy constraints. Consensus-based optimization consists of a set of computational nodes arranged in a graph, each having a local objective that depends on their local data, where in every step nodes take a linear combination of their neighbors' messages, as well as taking a new gradient step. Since the algorithm requires exchanging messages that depend on local data, private information gets leaked at every …
In this paper, we propose a fully distributed algorithm for second-order continuous-time multi-agent systems to solve the distributed optimization problem. The global objective function is a sum of private cost functions associated with the individual agents and the interaction between agents is described by a weighted undirected graph. We show the exponential convergence of the proposed algorithm if the underlying graph is connected, each private cost function is locally gradient-Lipschitz-continuous, and the global objective function is restricted strongly convex with respect to the global m…
We present an optimization framework for solving multi-agent nonlinear programs subject to inequality constraints while keeping the agents' state trajectories private. Each agent has an objective function depending only upon its own state and the agents are collectively subject to global constraints. The agents do not directly communicate with each other but instead route messages through a trusted cloud computer. The cloud computer adds noise to data being sent to the agents in accordance with the framework of differential privacy in order to keep each agent's state trajectory private from al…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper considers a distributed multi-agent optimization problem, with the global objective consisting of the sum of local objective functions of the agents. The agents solve the optimization problem using local computation and communication between adjacent agents in the network. We present two randomized iterative algorithms for distributed optimization. To improve privacy, our algorithms add "structured" randomization to the information exchanged between the agents. We prove deterministic correctness (in every execution) of the proposed algorithms despite the information being perturbed …
We consider cooperative multi-agent consensus optimization problems over an undirected network of agents, where only those agents connected by an edge can directly communicate. The objective is to minimize the sum of agent-specific composite convex functions over agent-specific private conic constraint sets; hence, the optimal consensus decision should lie in the intersection of these private sets. We provide convergence rates both in sub-optimality, infeasibility and consensus violation; examine the effect of underlying network topology on the convergence rates of the proposed decentralized a…
