Parabolic optimal control problems with control constraints are generally challenging, from either theoretical analysis or algorithmic design perspectives. Conceptually, the well-known alternating direction method of multipliers (ADMM) can be directly applied to such a problem. An attractive advantage of this direct ADMM application is that the control constraint can be untied from the parabolic PDE constraint; these two inherently different constraints thus can be treated individually in iterations. At each iteration of the ADMM, the main computation is for solving an unconstrained parabolic optimal control problem. Because of its high dimensionality after discretization, the unconstrained parabolic optimal control problem at each iteration can be solved only inexactly by implementing certain numerical scheme internally and thus a two-layer nested iterative scheme is required. It then becomes important to find an easily implementable and efficient inexactness criterion to execute the internal iterations, and to prove the overall convergence rigorously for the resulting two-layer nested iterative scheme. To implement the ADMM efficiently, we propose an inexactness criterion that is independent of the mesh size of the involved discretization, and it can be executed automatically with no need to set empirically perceived constant accuracy a prior. The inexactness criterion turns out to allow us to solve the resulting unconstrained optimal control problems to medium or even low accuracy and thus saves computation significantly, yet convergence of the overall two-layer nested iterative scheme can be still guaranteed rigorously. Efficiency of this ADMM implementation is promisingly validated by preliminary numerical results. Our methodology can also be extended to a range of optimal control problems constrained by other linear PDEs such as elliptic equations and hyperbolic equations.
The alternating direction method of multipliers (ADMM) is a powerful operator splitting technique for solving structured convex optimization problems. Due to its relatively low per-iteration computational cost and ability to exploit sparsity in the problem data, it is particularly suitable for large-scale optimization. However, the method may still take prohibitively long to compute solutions to very large problem instances. Although ADMM is known to be parallelizable, this feature is rarely exploited in real implementations. In this paper we exploit the parallel computing architecture of a gr…
An alternating direction method of multipliers (ADMM) solver is described for optimal resource allocation problems with separable convex quadratic costs and constraints and linear coupling constraints. We describe a parallel implementation of the solver on a graphics processing unit (GPU) using a bespoke quartic function minimizer. An application to robust optimal energy management in hybrid electric vehicles is described, and the results of numerical simulations comparing the computation times of the parallel GPU implementation with those of an equivalent serial implementation are presented.
Non-convex quadratically constrained quadratic programming (QCQP) problems have numerous applications in signal processing, machine learning, and wireless communications, albeit the general QCQP is NP-hard, and several interesting special cases are NP-hard as well. This paper proposes a new algorithm for general QCQP. The problem is first reformulated in consensus optimization form, to which the alternating direction method of multipliers (ADMM) can be applied. The reformulation is done in such a way that each of the sub-problems is a QCQP with only one constraint (QCQP-1), which is efficientl…
This paper proposes a two-level distributed algorithmic framework for solving the AC optimal power flow (OPF) problem with convergence guarantees. The presence of highly nonconvex constraints in OPF poses significant challenges to distributed algorithms based on the alternating direction method of multipliers (ADMM). In particular, convergence is not provably guaranteed for nonconvex network optimization problems like AC OPF. In order to overcome this difficulty, we propose a new distributed reformulation for AC OPF and a two-level ADMM algorithm that goes beyond the standard framework of ADMM…
This paper aims to present a fairly accessible generalization of several symmetric Gauss-Seidel decomposition based multi-block proximal alternating direction methods of multipliers (ADMMs) for convex composite optimization problems. The proposed method unifies and refines many constructive techniques that were separately developed for the computational efficiency of multi-block ADMM-type algorithms. Specifically, the majorized augmented Lagrangian functions, the indefinite proximal terms, the inexact symmetric Gauss-Seidel decomposition theorem, the tolerance criteria of approximately solving…
We consider a model predictive control (MPC) setting, where we use the alternating direction method of multipliers (ADMM) to exploit problem structure. We take advantage of interacting components in the controlled system by decomposing its dynamics with virtual subsystems and virtual inputs. We introduce subsystem-individual penalty parameters together with optimal selection techniques. Further, we propose a novel measure of system structure, which we call separation tendency. For a sufficiently structured system, the resulting structure-exploiting method has the following characteristics: (i)…
In this paper, we show that for a class of linearly constrained convex composite optimization problems, an (inexact) symmetric Gauss-Seidel based majorized multi-block proximal alternating direction method of multipliers (ADMM) is equivalent to an {\em inexact} proximal augmented Lagrangian method (ALM). This equivalence not only provides new perspectives for understanding some ADMM-type algorithms but also supplies meaningful guidelines on implementing them to achieve better computational efficiency. Even for the two-block case, a by-product of this equivalence is the convergence of the whole…
This article presents a sparse, low-memory footprint optimization algorithm for the implementation of the model predictive control (MPC) for tracking formulation in embedded systems. This MPC formulation has several advantages over standard MPC formulations, such as an increased domain of attraction and guaranteed recursive feasibility even in the event of a sudden reference change. However, this comes at the expense of the addition of a small amount of decision variables to the MPC's optimization problem that complicates the structure of its matrices. We propose a sparse optimization algorith…
In this paper, elliptic control problems with pointwise box constraints on the state is considered, where the corresponding Lagrange multipliers in general only represent regular Borel measure functions. To tackle this difficulty, the Lavrentiev regularization is employed to deal with the state constraints. To numerically discretize the resulted problem, since the weakness of variational discretization in numerical implementation, full piecewise linear finite element discretization is employed. Estimation of the error produced by regularization and discretization is done. The error order of fu…
National Renewable Energy Laboratory（国家可再生能源实验室）；University of Colorado at Boulder（科罗拉多大学博尔德分校）
We propose a novel data-driven method to accelerate the convergence of Alternating Direction Method of Multipliers (ADMM) for solving distributed DC optimal power flow (DC-OPF) where lines are shared between independent network partitions. Using previous observations of ADMM trajectories for a given system under varying load, the method trains a recurrent neural network (RNN) to predict the converged values of dual and consensus variables. Given a new realization of system load, a small number of initial ADMM iterations is taken as input to infer the converged values and directly inject them i…
Phase retrieval has important applications in optical imaging, communications and sensing. Lifting the dimensionality of the problem allows phase retrieval to be approximated as a convex optimization problem in a higher-dimensional space. Convex optimization-based phase retrieval has been shown to yield high accuracy, yet its low-complexity implementation has not been explored. In this paper, we study three fundamental approaches for its low-complexity implementation: the projected gradient method, the Nesterov accelerated gradient method, and the alternating direction method of multipliers (A…
This work investigates the theoretical performance of the alternating-direction method of multipliers (ADMM) as it applies to nonconvex optimization problems, and in particular, problems with nonconvex constraint sets. The alternating direction method of multipliers is an optimization method that has largely been analyzed for convex problems. The ultimate goal is to assess what kind of theoretical convergence properties the method has in the nonconvex case, and to this end, theoretical contributions are two-fold. First, this work analyzes the method with local solution of the ADMM subproblems,…
In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex) regularization function. The proposed algorithm incorporates ideas from several existing approaches such as alternate direction method of multipliers (ADMM), successive convex approximation (SCA), distributed and asynchronous algorithms, and inexact gradient methods. Different from a number of existing approaches, however, the proposed framework is flexible…
Numerical modeling and open-source implementation of variational partition-of-unity localizations of space-time dual-weighted residual estimators for parabolic problems
Design of optimal distributed linear feedback controllers to achieve a desired aggregate behavior, while simultaneously satisfying state and input constraints, is a challenging but important problem in many applications. System level synthesis is a recent technique which has been used to reparametrize the optimal control problem as a convex program. Prior work on system level synthesis with state and input constraints has included closed-loop finite impulse response and locality constraints or, in the case where these constraints were lifted using a simple pole approximation, only a centralize…
An oblique projections based feedback stabilizability result in the literature is extended to a larger class of reaction-convection terms. A discussion is presented including a comparison between explicit oblique projections base feedback controls and Riccati based feedback controls. Advantages and limitations of each type of feedback are addressed as well as their finite-elements implementation. Results of numerical simulations are presented comparing their stabilizing performances for the case of time-periodic dynamics. It is shown that the solution of the periodic Riccati based feedback can…
We present two efficient numerical methods for susceptibility artifact correction applicable in Echo Planar Imaging (EPI), an ultra fast Magnetic Resonance Imaging (MRI) technique widely used in clinical applications. Both methods address a major practical drawback of EPI, the so-called susceptibility artifacts, which consist of geometrical transformations and intensity modulations. We consider a tailored variational image registration problem that is based on a physical distortion model and aims at minimizing the distance of two oppositely distorted images subject to invertibility constraints…
We present a hardware-based implementation of Linear Program (LP) decoding for binary linear codes. LP decoding frames error-correction as an optimization problem. In contrast, variants of Belief Propagation (BP) decoding frame error-correction as a problem of graphical inference. LP decoding has several advantages over BP-based methods, including convergence guarantees and better error-rate performance in high-reliability channels. The latter makes LP decoding attractive for optical transport and storage applications. However, LP decoding, when implemented with general solvers, does not scale…
We study constant input delay compensation by using finite-dimensional observer-based controllers in the case of the 1D heat equation. We consider Neumann actuation with nonlocal measurement and employ modal decomposition with $N+1$ modes in the observer. We introduce a chain of $M$ sub-predictors that leads to a closed-loop ODE system coupled with infinite-dimensional tail. Given an input delay $r$, we present LMI stability conditions for finding $M$ and $N$ and the resulting exponential decay rate and prove that the LMIs are always feasible for any $r$. We also consider a classical observer-…
The increasing presence of large-scale distributed systems highlights the need for scalable control strategies where only local communication is required. Moreover, in safety-critical systems it is imperative that such control strategies handle constraints in the presence of disturbances. In response to this need, we present the Distributed and Localized Model Predictive Control (DLMPC) algorithm for large-scale linear systems. DLMPC is a distributed closed-loop model predictive control (MPC) scheme wherein only local state and model information needs to be exchanged between subsystems for the…
We propose and investigate a general class of discrete time and finite state space mean field game (MFG) problems with potential structure. Our model incorporates interactions through a congestion term and a price variable. It also allows hard constraints on the distribution of the agents. We analyze the connection between the MFG problem and two optimal control problems in duality. We present two families of numerical methods and detail their implementation: (i) primal-dual proximal methods (and their extension with nonlinear proximity operators), (ii) the alternating direction method of mult…
We consider a linear non-local heat equation in a bounded domain $Ω\subset\mathbb{R}^d$, $d\geq 1$, with Dirichlet boundary conditions, where the non-locality is given by the presence of an integral kernel. Motivated by several applications in biological systems, in the present paper we study some optimal control problems from a theoretical and numerical point of view. In particular, we will employ the classical low-regret approach of J.-L. Lions for treating the problem of incomplete data and provide a simple computational implementation of the method. The effectiveness of the results are ill…
SDPNAL+ is a {\sc Matlab} software package that implements an augmented Lagrangian based method to solve large scale semidefinite programming problems with bound constraints. The implementation was initially based on a majorized semismooth Newton-CG augmented Lagrangian method, here we designed it within an inexact symmetric Gauss-Seidel based semi-proximal ADMM/ALM (alternating direction method of multipliers/augmented Lagrangian method) framework for the purpose of deriving simpler stopping conditions and closing the gap between the practical implementation of the algorithm and the theoretic…
In this paper we consider a constrained parabolic optimal control problem. The cost functional is quadratic and it combines the distance of the trajectory of the system from the desired evolution profile together with the cost of a control. The constraint is given by a term measuring the distance between the final state and the desired state towards which the solution should be steered. The control enters the system through the initial condition. We present a geometric analysis of this problem and provide a closed-form expression for the solution. This approach allows us to present the sensiti…
Mean-variance portfolio optimization problems often involve separable nonconvex terms, including penalties on capital gains, integer share constraints, and minimum position and trade sizes. We propose a heuristic algorithm for such problems based on the alternating direction method of multipliers (ADMM). This method allows for solve times in tens to hundreds of milliseconds with around 1000 securities and 100 risk factors. We also obtain a bound on the achievable performance. Our heuristic and bound are both derived from similar results for other optimization problems with a separable objectiv…
Many modern statistically efficient methods come with tremendous computational challenges, often leading to large-scale optimisation problems. In this work, we examine such computational issues for recently developed estimation methods in nonparametric regression with a specific view on image denoising. We consider in particular certain variational multiscale estimators which are statistically optimal in minimax sense, yet computationally intensive. Such an estimator is computed as the minimiser of a smoothness functional (e.g., TV norm) over the class of all estimators such that none of its c…
Solving the power flow problem in a distributed fashion empowers different grid operators to compute the overall grid state without having to share grid models-this is a practical problem to which industry does not have off-the-shelf answers. In cooperation with a German transmission system operator we propose two physically consistent problem formulations (feasibility, least-squares) amenable to two solution methods from distributed optimization (the Alternating direction method of multipliers (ADMM), and the Augmented Lagrangian based Alternating Direction Inexact Newton method (Aladin)); wi…
