Alternating-Current Optimal Power Flow (AC-OPF) is framed as a NP-hard non-convex optimization problem that solves for the most economical dispatch of grid generation given the AC-network and device constraints. Although there are no standard methodologies for obtaining the global optimum for the problem, there is considerable interest from planning and operational engineers in finding a local optimum. Nonetheless, solving for the local optima of a large AC-OPF problem is challenging and time-intensive, as none of the leading non-linear optimization toolboxes can provide any timely guarantees of convergence. To provide robust local convergence for large complex systems, we introduce a homotopy-based approach that solves a sequence of primal-dual interior point problems. We utilize the physics of the grid to develop the proposed homotopy method and demonstrate the efficacy of this approach on U.S. Eastern Interconnection sized test networks.
Algorithms for solving nonconvex, nonsmooth, finite-sum optimization problems are proposed and tested. In particular, the algorithms are proposed and tested in the context of an optimization problem formulation arising in semi-supervised machine learning. The common feature of all algorithms is that they employ an incremental quasi-Newton (IQN) strategy, specifically an incremental BFGS (IBFGS) strategy. One applies an IBFGS strategy to the problem directly, whereas the others apply an IBFGS strategy to a difference-of-convex reformulation, smoothed approximation, or (strongly) convex local ap…
The generalized Nash equilibrium problem (GNEP) is a kind of game to find strategies for a group of players such that each player's objective function is optimized. Solutions for GNEPs are called generalized Nash equilibria (GNEs). In this paper, we propose a numerical method for finding GNEs of GNEPs of polynomials based on the polyhedral homotopy continuation and the Moment-SOS hierarchy of semidefinite relaxations. We show that our method can find all GNEs if they exist, or detect the nonexistence of GNEs, under some genericity assumptions. Some numerical experiments are made to demonstrate…
Incremental methods are widely utilized for solving finite-sum optimization problems in machine learning and signal processing. In this paper, we study a family of incremental methods -- including incremental subgradient, incremental proximal point, and incremental prox-linear methods -- for solving weakly convex optimization problems. Such a problem class covers many nonsmooth nonconvex instances that arise in engineering fields. We show that the three said incremental methods have an iteration complexity of $O(\varepsilon^{-4})$ for driving a natural stationarity measure to below $\varepsilo…
We propose a new primal-dual homotopy smoothing algorithm for a linearly constrained convex program, where neither the primal nor the dual function has to be smooth or strongly convex. The best known iteration complexity solving such a non-smooth problem is $\mathcal{O}(\varepsilon^{-1})$. In this paper, we show that by leveraging a local error bound condition on the dual function, the proposed algorithm can achieve a better primal convergence time of $\mathcal{O}\left(\varepsilon^{-2/(2+β)}\log_2(\varepsilon^{-1})\right)$, where $β\in(0,1]$ is a local error bound parameter. As an example appl…
This paper presents a new method for solving a class of nonlinear optimal control problems with a quadratic performance index. In this method, first the original optimal control problem is transformed into a nonlinear two-point boundary value problem (TPBVP) via the Pontryagin's maximum principle. Then, using the Homotopy Perturbation Method (HPM) and introducing a convex homotopy in topologic space, the nonlinear TPBVP is transformed into a sequence of linear time-invariant TPBVP's. By solving the presented linear TPBVP sequence in a recursive manner, the optimal control law and the optimal t…
This paper presents a computationally efficient optimization algorithm for solving nonconvex optimal control problems that involve discrete logic constraints. Traditional solution methods for these constraints require binary variables and mixed-integer programming, which is prohibitively slow and computationally expensive. This paper targets a fast solution that is capable of real-time implementation onboard spacecraft. To do so, a novel algorithm is developed that blends sequential convex programming and numerical continuation into a single iterative solution process. Inside the algorithm, di…
In this paper, we develop a drift homotopy implicit particle filter method. The methodology of our approach is to adopt the concept of drift homotopy in the resampling procedure of the particle filter method for solving the nonlinear filtering problem, and we introduce an implicit particle filter method to improve the efficiency of the drift homotopy resampling procedure. Numerical experiments are carried out to demonstrate the effectiveness and efficiency of our drift homotopy implicit particle filter.
- In this paper we introduce a new method to solve fixed-delay optimal control problems which exploits numerical homotopy procedures. It is known that solving this kind of problems via indirect methods is complex and computationally demanding because their implementation is faced with two difficulties: the extremal equations are of mixed type, and besides, the shooting method has to be carefully initialized. Here, starting from the solution of the non-delayed version of the optimal control problem, the delay is introduced by numerical homotopy methods. Convergence results, which ensure the eff…
A method for solving zero-finding problems is developed by tracking homotopy paths, which define connecting channels between an auxiliary problem and the objective problem. Current algorithms' success highly relies on empirical knowledge, due to manually, inherently selected homotopy paths. This work introduces a homotopy method based on the Theory of Functional Connections (TFC). The TFC-based method implicitly defines infinite homotopy paths, from which the most promising ones are selected. A two-layer continuation algorithm is devised, where the first layer tracks the homotopy path by monot…
In this paper we propose a method that uses Lagrange multipliers and numerical algebraic geometry to find all critical points, and therefore globally solve, polynomial optimization problems. We design a polyhedral homotopy algorithm that explicitly constructs an optimal start system, circumventing the typical bottleneck associated with polyhedral homotopy algorithms. The correctness of our algorithm follows from intersection theoretic computations of the algebraic degree of polynomial optimization programs and relies on explicitly solving the tropicalization of a corresponding Lagrange system.…
$ \ell_1 $-regularized linear inverse problems are frequently used in signal processing, image analysis, and statistics. The correct choice of the regularization parameter $ t \in \mathbb{R}_{\geq 0} $ is a delicate issue. Instead of solving the variational problem for a fixed parameter, the idea of the homotopy method is to compute a complete solution path $ u(t) $ as a function of $ t $. In a celebrated paper by Osborne, Presnell, and Turlach, it has been shown that the computational cost of this approach is often comparable to the cost of solving the corresponding least squares problem. The…
In this paper we propose a primal-dual homotopy method for $\ell_1$-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path for the dual problem. We show that by solving a small linear program, one can jump to the next primal break point and then, solving another small linear program, a new optimal dual solution is calculate…
This paper suggests two novel ideas to develop new proximal variable-metric methods for solving a class of composite convex optimization problems. The first idea is a new parameterization of the optimality condition which allows us to develop a class of homotopy proximal variable-metric methods. We show that under appropriate assumptions such as strong convexity-type and smoothness, or self-concordance, our new schemes can achieve finite global iteration-complexity bounds. Our second idea is a primal-dual-primal framework for proximal-Newton methods which can lead to some useful computational …
A warm start method is developed for efficiently solving complex chance constrained optimal control problems. The warm start method addresses the computational challenges of solving chance constrained optimal control problems using biased kernel density estimators and Legendre-Gauss-Radau collocation with an $hp$ adaptive mesh refinement method. To address the computational challenges, the warm start method improves both the starting point for the chance constrained optimal control problem, as well as the efficiency of cycling through mesh refinement iterations. The improvement is accomplished…
In this paper, based on a $Q$-linear convergence analysis and an estimate of the linear convergence factor of the proximal point (PP) algorithm for solving box constrained quadratic programming (BQP) problems, an accelerated proximal point (APP) algorithm for solving BQP problems is presented. To solve the strictly convex BQP problems in each step of the APP algorithm, an efficient homotopy method, which tracks the solution path of a parametric quadratic program, is given. The algorithm with APP algorithm as outer iteration and the homotopy method as inner iteration is named by APP-Hom. The in…
Accelerated proximal gradient methods have recently been developed for solving quasi-static incremental problems of elastoplastic analysis with some different yield criteria. It has been demonstrated through numerical experiments that these methods can outperform conventional optimization-based approaches in computational plasticity. However, in literature these algorithms are described individually for specific yield criteria, and hence there exists no guide for application of the algorithms to other yield criteria. This short paper presents a general form of algorithm design, independent of …
The existing machine learning algorithms for minimizing the convex function over a closed convex set suffer from slow convergence because their learning rates must be determined before running them. This paper proposes two machine learning algorithms incorporating the line search method, which automatically and algorithmically finds appropriate learning rates at run-time. One algorithm is based on the incremental subgradient algorithm, which sequentially and cyclically uses each of the parts of the objective function; the other is based on the parallel subgradient algorithm, which uses parts i…
The increasing penetration of renewable and distributed energy resources in distribution networks calls for real-time and distributed voltage control. In this paper we investigate local Volt/VAR control with a general class of control functions, and show that the power system dynamics with non-incremental local voltage control can be seen as distributed algorithm for solving a well-defined optimization problem (reverse engineering). The reverse engineering further reveals a fundamental limitation of the non-incremental voltage control: the convergence condition is restrictive and prevents bett…
In this paper we introduce a new procedure to solve nonlinear optimal control problems with delays which exploits indirect methods combined with numerical homotopy procedures. It is known that solving this kind of problems via indirect methods (which arise from the Pontrya-gin Maximum Principle) is complex and computationally demanding because their implementation is faced to two main difficulties: the extremal equations involve forward and backward terms, and besides, the related shooting method has to be carefully initialized. Here, starting from the solution of the non-delayed version of th…
Incremental Gauss--Newton Methods with Superlinear Convergence Rates
School of Data Science, Fudan University（复旦大学数据科学学院）；National University of Singapore（新加坡国立大学）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）
In recent years, important progress has been made in applying methods and techniques of convex optimization to many fields of applications such as location science, engineering, computational statistics, and computer science. In this paper, we present some simple algorithms for solving a number of new models of facility location involving sets which generalize the classical Fermat-Torricelli problem and the smallest enclosing circle problem. The general nondifferentiability of the models prevents us from applying gradient-type algorithms, so our approach is to use subgradient-type algorithms i…
This paper presents a new and straightforward procedure for solving bilinear quadratic optimal control problem. In this method, first the original optimal control problem is transformed into a nonlinear twopoint boundary value problem (TPBVP) via the Pontryagin's maximum principle. Then, the nonlinear TPBVP is transformed into a sequence of linear time-invariant TPBVPs using the homotopy perturbation method (HPM) and introducing a convex homotopy in topologic space. Solving the latter problems through an iterative process yields the optimal control law and optimal trajectory in the form of inf…
In this paper, we consider a class of finite-sum convex optimization problems whose objective function is given by the summation of $m$ ($\ge 1$) smooth components together with some other relatively simple terms. We first introduce a deterministic primal-dual gradient (PDG) method that can achieve the optimal black-box iteration complexity for solving these composite optimization problems using a primal-dual termination criterion. Our major contribution is to develop a randomized primal-dual gradient (RPDG) method, which needs to compute the gradient of only one randomly selected smooth compo…
The problem of minimizing the sum of nonsmooth, convex objective functions defined on a real Hilbert space over the intersection of fixed point sets of nonexpansive mappings, onto which the projections cannot be efficiently computed, is considered. The use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem. With the focus on fixed point approximation techniques, two algorithms are devised for solving the problem. One blends an incremental subgradient method, which is a useful algor…
A conditional gradient homotopy method with applications to Semidefinite Programming
Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机学研究所）；Zuse Institute Berlin（柏林楚泽研究所）；Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology（以色列理工学院数据与决策科学学院）；Department of Mathematics（数学系）
University of Freiburg（弗莱堡大学）；University of North Carolina（北卡罗来纳大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）
First-order stochastic methods for solving large-scale non-convex optimization problems are widely used in many big-data applications, e.g. training deep neural networks as well as other complex and potentially non-convex machine learning models. Their inexpensive iterations generally come together with slow global convergence rate (mostly sublinear), leading to the necessity of carrying out a very high number of iterations before the iterates reach a neighborhood of a minimizer. In this work, we present a first-order stochastic algorithm based on a combination of homotopy methods and SGD, cal…
The alternating direction method of multipliers (ADMM) has been recently recognized as a promising optimizer for large-scale machine learning models. However, there are very few results studying ADMM from the aspect of communication costs, especially jointly with privacy preservation, which are critical for distributed learning. We investigate the communication efficiency and privacy-preservation of ADMM by solving the consensus optimization problem over decentralized networks. Since walk algorithms can reduce communication load, we first propose incremental ADMM (I-ADMM) based on the walk alg…
The linear convex log-homotopy has been used in the derivation of particle flow filters. One natural question is whether it is beneficial to consider other forms of homotopy. We revisit this question by considering a general linear form of log-homotopy for which we derive particle flow filters, validate the distribution of flows, and obtain conditions for the stability of particle flows. We then formulate the problem of stiffness mitigation as an optimal control problem by minimizing the condition number of the Hessian matrix of the posterior density function. The optimal homotopy can be effic…
University of Pennsylvania（宾夕法尼亚大学）；Rutgers University（罗格斯大学）
Recently, there has been growing interest in developing optimization methods for solving large-scale machine learning problems. Most of these problems boil down to the problem of minimizing an average of a finite set of smooth and strongly convex functions where the number of functions $n$ is large. Gradient descent method (GD) is successful in minimizing convex problems at a fast linear rate; however, it is not applicable to the considered large-scale optimization setting because of the high computational complexity. Incremental methods resolve this drawback of gradient methods by replacing t…
We consider solving the $\ell_1$-regularized least-squares ($\ell_1$-LS) problem in the context of sparse recovery, for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a rather slow convergence rate. Nevertheless, when the solution is sparse, it often exhibits fast linear convergence in the final stage. We exploit the local linear convergence using a homotopy continuation strategy, i.e., we solve the $\ell_1$-LS problem for a sequence of decreasin…
Georgia Institute of Technology（佐治亚理工学院）；Tsinghua University（清华大学）；IBM Almaden Research Center（IBM阿尔马登研究中心）
We propose a novel randomized incremental gradient algorithm, namely, VAriance-Reduced Accelerated Gradient (Varag), for finite-sum optimization. Equipped with a unified step-size policy that adjusts itself to the value of the condition number, Varag exhibits the unified optimal rates of convergence for solving smooth convex finite-sum problems directly regardless of their strong convexity. Moreover, Varag is the first accelerated randomized incremental gradient method that benefits from the strong convexity of the data-fidelity term to achieve the optimal linear convergence. It also establish…
We propose the new notion of Visco-Energetic solutions to rate-independent systems $(X,\mathcal E,\mathsf d)$ driven by a time dependent energy $\mathcal E$ and a dissipation quasi-distance $\mathsf d$ in a general metric-topological space $X$. As for the classic Energetic approach, solutions can be obtained by solving a modified time Incremental Minimization Scheme, where at each step the dissipation (quasi-)distance $\mathsf d$ is incremented by a viscous correction $δ$ (e.g.~proportional to the square of the distance $\mathsf d$), which penalizes far distance jumps by inducing a localized v…
