This paper investigates distributed online convex optimization in the presence of an aggregative variable without any global/central coordinators over a multi-agent network, where each individual agent is only able to access partial information of time-varying global loss functions, thus requiring local information exchanges between neighboring agents. Motivated by many applications in reality, the considered local loss functions depend not only on their own decision variables, but also on an aggregative variable, such as the average of all decision variables. To handle this problem, an Online Distributed Gradient Tracking algorithm (O-DGT) is proposed with exact gradient information and it is shown that the dynamic regret is upper bounded by three terms: a sublinear term, a path variation term, and a gradient variation term. Meanwhile, the O-DGT algorithm is also analyzed with stochastic/noisy gradients, showing that the expected dynamic regret has the same upper bound as the exact gradient case. To our best knowledge, this paper is the first to study online convex optimization in the presence of an aggregative variable, which enjoys new characteristics in comparison with the conventional scenario without the aggregative variable. Finally, a numerical experiment is provided to corroborate the obtained theoretical results.
This paper designs a distributed stochastic annealing algorithm for non-convex cooperative aggregative games, whose agents' cost functions not only depend on agents' own decision variables but also rely on the sum of agents' decision variables. To seek the the social optimum of cooperative aggregative games, a distributed stochastic annealing algorithm is proposed, where the local cost functions are non-convex and the communication topology between agents is time varying. The weak convergence to the social optimum of the algorithm is further analyzed. A numerical example is given to illustrate…
This paper proposes a new framework for distributed optimization, called distributed aggregative optimization, which allows local objective functions to be dependent not only on their own decision variables, but also on the average of summable functions of decision variables of all other agents. To handle this problem, a distributed algorithm, called distributed gradient tracking (DGT), is proposed and analyzed, where the global objective function is strongly convex, and the communication graph is balanced and strongly connected. It is shown that the algorithm can converge to the optimal varia…
In this paper, we focus on solving a distributed convex aggregative optimization problem in a network, where each agent has its own cost function which depends not only on its own decision variables but also on the aggregated function of all agents' decision variables. The decision variable is constrained within a feasible set. In order to minimize the sum of the cost functions when each agent only knows its local cost function, we propose a distributed Frank-Wolfe algorithm based on gradient tracking for the aggregative optimization problem where each node maintains two estimates, namely an e…
The online optimization problem with non-convex loss functions over a closed convex set, coupled with a set of inequality (possibly non-convex) constraints is a challenging online learning problem. A proximal method of multipliers with quadratic approximations (named as OPMM) is presented to solve this online non-convex optimization with long term constraints. Regrets of the violation of Karush-Kuhn-Tucker conditions of OPMM for solving online non-convex optimization problems are analyzed. Under mild conditions, it is shown that this algorithm exhibits ${\cO}(T^{-1/8})$ Lagrangian gradient vio…
Recently, there were introduced important classes of relatively smooth, relatively continuous, and relatively strongly convex optimization problems. These concepts have significantly expanded the class of problems for which optimal complexity estimates of gradient-type methods in high-dimensional spaces take place. Basing on some recent works devoted to online optimization (regret minimization) problems with both relatively Lipschitz continuous and relatively strongly convex objective function, we introduce algorithms for solving the strongly convex optimization problem with inequality constra…
Gradient-tracking (GT) based decentralized methods have emerged as an effective and viable alternative method to decentralized (stochastic) gradient descent (DSGD) when solving distributed online stochastic optimization problems. Initial studies of GT methods implied that GT methods have worse network dependent rate than DSGD, contradicting experimental results. This dilemma has recently been resolved, and tighter rates for GT methods have been established, which improves upon DSGD. In this work, we establish more enhanced rates for GT methods under the online stochastic convex settings. We pr…
This paper considers distributed online convex constrained optimization, in which various agents in a multi-agent system cooperate to minimize a global cost function through communicating with neighbors over a time-varying network. When the constraint set of optimization problem is high-dimensional and complicated, the computational cost of the projection operation often becomes prohibitive. To handle this problem, we develop a distributed online Frank-Wolfe optimization algorithm combining with gradient tracking technique. We rigorously establish the dynamic regret bound of the proposed optim…
Many techniques for online optimization problems involve making decisions based solely on presently available information: fewer works take advantage of potential predictions. In this paper, we discuss the problem of online convex optimization for parametrizable objectives, i.e. optimization problems that depend solely on the value of a parameter at a given time. We introduce a new regularity for dynamic regret based on the accuracy of predicted values of the parameters and show that, under mild assumptions, accurate prediction can yield tighter bounds on dynamic regret. Inspired by recent adv…
We study the problem of networked online convex optimization, where each agent individually decides on an action at every time step and agents cooperatively seek to minimize the total global cost over a finite horizon. The global cost is made up of three types of local costs: convex node costs, temporal interaction costs, and spatial interaction costs. In deciding their individual action at each time, an agent has access to predictions of local cost functions for the next $k$ time steps in an $r$-hop neighborhood. Our work proposes a novel online algorithm, Localized Predictive Control (LPC), …
We present the online Newton's method, a single-step second-order method for online nonconvex optimization. We analyze its performance and obtain a dynamic regret bound that is linear in the cumulative variation between round optima. We show that if the variation between round optima is limited, the method leads to a constant regret bound. In the general case, the online Newton's method outperforms online convex optimization algorithms for convex functions and performs similarly to a specialized algorithm for strongly convex functions. We simulate the performance of the online Newton's method …
In this paper, we consider the problem of distributed online convex optimization, where a group of agents collaborate to track the global minimizers of a sum of time-varying objective functions in an online manner. Specifically, we propose a novel distributed online gradient descent algorithm that relies on an online adaptation of the gradient tracking technique used in static optimization. We show that the dynamic regret bound of this algorithm has no explicit dependence on the time horizon and, therefore, can be tighter than existing bounds especially for problems with long horizons. Our bou…
We propose a data-driven online convex optimization algorithm for controlling dynamical systems. In particular, the control scheme makes use of an initially measured input-output trajectory and behavioral systems theory which enable it to handle unknown discrete-time linear time-invariant systems as well as a priori unknown time-varying cost functions. Further, only output feedback instead of full state measurements is required for the proposed approach. Analysis of the closed loop's performance reveals that the algorithm achieves sublinear regret if the variation of the cost functions is subl…
This paper studies the problem of controlling linear dynamical systems subject to point-wise-in-time constraints. We present an algorithm similar to online gradient descent, that can handle time-varying and a priori unknown convex cost functions while restraining the system states and inputs to polytopic constraint sets. Analysis of the algorithm's performance, measured by dynamic regret, reveals that sublinear regret is achieved if the variation of the cost functions is sublinear in time. Finally, we present a simple example to illustrate implementation details as well as the algorithm's perf…
This paper studies the online convex optimization problem by using an Online Continuous-Time Nesterov Accelerated Gradient method (OCT-NAG). We show that the continuous-time dynamics generated by the online version of the Bregman Lagrangian achieves a constant static regret $\frac{c}σ$ independent of $T$, provided that some boundedness assumptions on the objective functions and optimal solutions hold. To the best of the authors' knowledge, this is the lowest static regret in the literature (lower than $O(\text{log}(T))$). We further show that under the same assumptions, the dynamic regret of t…
This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging time-varying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constraints. Departing from existing batch and feed-forward optimization approaches, the design of the algorithms capitalizes on an online implementation of primal-dual projected-gradient methods; the gradient…
This paper investigates online composite optimization in dynamic environments, where each objective or loss function contains a time-varying nondifferentiable regularizer. To resolve it, an online proximal gradient algorithm is studied for two distinct scenarios, including convex and strongly convex objectives without the smooth condition. In both scenarios, unlike most of works, an extended version of the conventional path variation is employed to bound the considered performance metric, i.e., dynamic regret. In the convex scenario, a bound $\mathcal{O}(\sqrt{T^{1-β}D_β(T)+T})$ is obtained wh…
Google AI Princeton（谷歌AI普林斯顿）；Princeton University（普林斯顿大学）
We study an algorithmic equivalence technique between non-convex gradient descent and convex mirror descent. We start by looking at a harder problem of regret minimization in online non-convex optimization. We show that under certain geometric and smoothness conditions, online gradient descent applied to non-convex functions is an approximation of online mirror descent applied to convex functions under reparameterization. In continuous time, the gradient flow with this reparameterization was shown to be exactly equivalent to continuous-time mirror descent by Amid and Warmuth 2020, but theory f…
University of California, Berkeley（加州大学伯克利分校）；University of Melbourne（墨尔本大学）；University of Toronto（多伦多大学）
We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the loads' behavior, can be used to improve on predictions made with only past observations. Specifically, we introduce an additional predictive step that follows the standard online convex optimization step when certain conditions on the estimated gradient and descent direction are met. We show that under these conditions and without any assumptions on the pre…
We investigate the problem of online convex optimization with unknown delays, in which the feedback of a decision arrives with an arbitrary delay. Previous studies have presented a delayed variant of online gradient descent (OGD), and achieved the regret bound of $O(\sqrt{T+D})$ by only utilizing the convexity condition, where $D$ is the sum of delays over $T$ rounds. In this paper, we further exploit the strong convexity to improve the regret bound. Specifically, we first extend the delayed variant of OGD for strongly convex functions, and establish a better regret bound of $O(d\log T)$, wher…
In this paper, we consider the problem of distributed online convex optimization, where a network of local agents aim to jointly optimize a convex function over a period of multiple time steps. The agents do not have any information about the future. Existing algorithms have established dynamic regret bounds that have explicit dependence on the number of time steps. In this work, we show that we can remove this dependence assuming that the local objective functions are strongly convex. More precisely, we propose a gradient tracking algorithm where agents jointly communicate and descend based o…
We extend the regret analysis of the online distributed weighted dual averaging (DWDA) algorithm [1] to the dynamic setting and provide the tightest dynamic regret bound known to date with respect to the time horizon for a distributed online convex optimization (OCO) algorithm. Our bound is linear in the cumulative difference between consecutive optima and does not depend explicitly on the time horizon. We use dynamic-online DWDA (D-ODWDA) and formulate a performance-guaranteed distributed online demand response approach for heating, ventilation, and air-conditioning (HVAC) systems of commerci…
It is known that the curvature of the feasible set in convex optimization allows for algorithms with better convergence rates, and there has been renewed interest in this topic both for offline as well as online problems. In this paper, leveraging results on geometry and convex analysis, we further our understanding of the role of curvature in optimization: - We first show the equivalence of two notions of curvature, namely strong convexity and gauge bodies, proving a conjecture of Abernethy et al. As a consequence, this show that the Frank-Wolfe-type method of Wang and Abernethy has accelerat…
Technion - Israel Institute of Technology（以色列理工学院）
Hoffman's classical result gives a bound on the distance of a point from a convex and compact polytope in terms of the magnitude of violation of the constraints. Recently, several results showed that Hoffman's bound can be used to derive strongly-convex-like rates for first-order methods for \textit{offline} convex optimization of curved, though not strongly convex, functions, over polyhedral sets. In this work, we use this classical result for the first time to obtain faster rates for \textit{online convex optimization} over polyhedral sets with curved convex, though not strongly convex, loss…
In this paper, we consider online convex optimization (OCO) with time-varying loss and constraint functions. Specifically, the decision maker chooses sequential decisions based only on past information, meantime the loss and constraint functions are revealed over time. We first develop a class of model-based augmented Lagrangian methods (MALM) for time-varying functional constrained OCO (without feedback delay). Under standard assumptions, we establish sublinear regret and sublinear constraint violation of MALM. Furthermore, we extend MALM to deal with time-varying functional constrained OCO w…
University of Colorado Boulder（科罗拉多大学博尔德分校）；IBM Research Ireland（IBM爱尔兰研究院）
This paper considers an online proximal-gradient method to track the minimizers of a composite convex function that may continuously evolve over time. The online proximal-gradient method is inexact, in the sense that: (i) it relies on an approximate first-order information of the smooth component of the cost; and, (ii) the proximal operator (with respect to the non-smooth term) may be computed only up to a certain precision. Under suitable assumptions, convergence of the error iterates is established for strongly convex cost functions. On the other hand, the dynamic regret is investigated when…
We study the problem of online learning (OL) from revealed preferences: a learner wishes to learn a non-strategic agent's private utility function through observing the agent's utility-maximizing actions in a changing environment. We adopt an online inverse optimization setup, where the learner observes a stream of agent's actions in an online fashion and the learning performance is measured by regret associated with a loss function. We first characterize a special but broad class of agent's utility functions, then utilize this structure in designing a new convex loss function. We establish th…
