We propose a novel framework for analyzing convergence rates of stochastic optimization algorithms with adaptive step sizes. This framework is based on analyzing properties of an underlying generic stochastic process, in particular by deriving a bound on the expected stopping time of this process. We utilize this framework to analyze the bounds on expected global convergence rates of a stochastic variant of a traditional trust region method, introduced in \cite{ChenMenickellyScheinberg2014}. While traditional trust region methods rely on exact computations of the gradient, Hessian and values of the objective function, this method assumes that these values are available up to some dynamically adjusted accuracy. Moreover, this accuracy is assumed to hold only with some sufficiently large, but fixed, probability, without any additional restrictions on the variance of the errors. This setting applies, for example, to standard stochastic optimization and machine learning formulations. Improving upon the analysis in \cite{ChenMenickellyScheinberg2014}, we show that the stochastic process defined by the algorithm satisfies the assumptions of our proposed general framework, with the stopping time defined as reaching accuracy $\|\nabla f(x)\|\leq ε$. The resulting bound for this stopping time is $O(ε^{-2})$, under the assumption of sufficiently accurate stochastic gradient, and is the first global complexity bound for a stochastic trust-region method. Finally, we apply the same framework to derive second order complexity bound under some additional assumptions.
We prove global convergence of a bundle trust region algorithm for non-smooth non-convex optimization, where cutting planes are generated by oracles respecting four basic rules. The benefit is that convergence theory applies to a large variety of methods encountered in practice. This includes in particular the method of downshifted tangents, for which previously no convergence result in the trust region framework was known. We also show that certain splitting techniques can be seen as special cases of bundle trust region techniques.
We propose a bundle trust-region algorithm to minimize locally Lipschitz functions which are potentially nonsmooth and nonconvex. We prove global convergence of our method and show by way of an example that the classical convergence argument in trust-region methods based on the Cauchy point fails in the nonsmooth setting. Our method is tested experimentally on three problems in automatic control.
This paper addresses some trust-region methods equipped with nonmonotone strategies for solving nonlinear unconstrained optimization problems. More specifically, the importance of using nonmonotone techniques in nonlinear optimization is motivated, then two new nonmonotone terms are proposed, and their combinations into the traditional trust-region framework are studied. The global convergence to first- and second-order stationary points and local superlinear and quadratic convergence rates for both algorithms are established. Numerical experiments on the \textsf{CUTEst} test collection of unc…
We propose a nonsmooth trust-region method for solving optimization problems with locally Lipschitz continuous functions, with application to problems constrained by variational inequalities of the second kind. Under suitable assumptions on the model functions, convergence of the general algorithm to a C-stationary point is verified. For variational inequality constrained problems, we are able to properly characterize the Bouligand subdifferential of the reduced cost function and, based on that, we propose a computable trust-region model which fulfills the convergence hypotheses of the general…
A novel trust region method for solving linearly constrained nonlinear programs is presented. The proposed technique is amenable to a distributed implementation, as its salient ingredient is an alternating projected gradient sweep in place of the Cauchy point computation. It is proven that the algorithm yields a sequence that globally converges to a critical point. As a result of some changes to the standard trust region method, namely a proximal regularisation of the trust region subproblem, it is shown that the local convergence rate is linear with an arbitrarily small ratio. Thus, convergen…
We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a (probably nonconvex) smooth function and a (probably nonsmooth) convex function. The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions. Therefore, the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods. When the accuracy of the model function at the solution of the subproblem is not sufficient, we a…
We consider descent methods for solving non-finite valued nonsmooth convex-composite optimization problems that employ Gauss-Newton subproblems to determine the iteration update. Specifically, we establish the global convergence properties for descent methods that use a backtracking line search, a weak Wolfe line search, or a trust-region update. All of these approaches are designed to exploit the structure associated with convex-composite problems.
In this paper, we propose a new and efficient nonmonotone adaptive trust region algorithm to solve unconstrained optimization problems. This algorithm incorporates two novelties: it benefits from a radius dependent shrinkage parameter for adjusting the trust region radius that avoids undesirable directions and it exploits a new strategy to prevent sudden increments of objective function values in nonmonotone trust region techniques. Global convergence of this algorithm is investigated under some mild conditions. Numerical experiments demonstrate the efficiency and robustness of the proposed al…
Motivated by TRACE algorithm [Curtis et al. 2017], we propose a trust region algorithm for finding second order stationary points of a linearly constrained non-convex optimization problem. We show the convergence of the proposed algorithm to (ε_g, ε_H)-second order stationary points in \widetilde{\mathcal{O}}(\max{ε_g^{-3/2}, ε_H^{-3}}) iterations. This iteration complexity is achieved for general linearly constrained optimization without cubic regularization of the objective function.
In many machine learning applications, one wants to learn the unknown objective and constraint functions of an optimization problem from available data and then apply some technique to attain a local optimizer of the learned model. This work considers Gaussian processes as global surrogate models and utilizes them in conjunction with derivative-free trust-region methods. It is well known that derivative-free trust-region methods converge globally---provided the surrogate model is probabilistically fully linear. We prove that \glspl{gp} are indeed probabilistically fully linear, thus resulting …
TRUST-TAEA: A trustworthiness-guided two-archive evolutionary algorithm with variable-grouping sparse search for large-scale multi-objective optimization
We present a new solution framework to solve the generalized trust region subproblem (GTRS) of minimizing a quadratic objective over a quadratic constraint. More specifically, we derive a convex quadratic reformulation (CQR) via minimizing a linear objective over two convex quadratic constraints for the GTRS. We show that an optimal solution of the GTRS can be recovered from an optimal solution of the CQR. We further prove that this CQR is equivalent to minimizing the maximum of the two convex quadratic functions derived from the CQR for the case under our investigation. Although the latter mi…
In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function $f(x)$, obtained from stochastic observations of the function or its gradient. Our method also utilizes estimates of function values to gauge progress that is being made. The convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with sufficiently high, but fixed, probability. Beyond these conditions, no assumptions are made on how these models and…
In this paper, a new alternating direction trust region method based on conic model is used to solve unconstrained optimization problems. By use of the alternating direction method, the new conic model trust region subproblem is solved by two steps in two orthogonal directions. This new idea overcomes the shortcomings of conic model subproblem which is difficult to solve. Then the global convergence of the method under some reasonable conditions is established. Numerical experiment shows that this method may be better than the dogleg method to solve the subproblem, especially for large-scale p…
Two-trust-region subproblem (TTRS), which is the minimization of a general quadratic function over the intersection of two full-dimensional ellipsoids, has been the subject of several recent research. In this paper, to solve TTRS, a hybrid of efficient algorithms for finding global and local-nonglobal minimizers of trust-region subproblem and the alternating direction method of multipliers (ADMM) is proposed. The convergence of the ADMM steps to the first order stationary condition is proved under certain conditions. On several classes of test problems, we compare the new algorithm with the re…
We develop a trust-region method for minimizing the sum of a smooth term $f$ and a nonsmooth term $h$), both of which can be nonconvex. Each iteration of our method minimizes a possibly nonconvex model of $f + h$ in a trust region. The model coincides with $f + h$ in value and subdifferential at the center. We establish global convergence to a first-order stationary point when $f$ satisfies a smoothness condition that holds, in particular, when it has Lipschitz-continuous gradient, and $h$ is proper and lower semi-continuous. The model of $h$ is required to be proper, lower-semi-continuous and…
Technion - Israel Institute of Technology（以色列理工学院）
Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspired by Conservative Policy Iteration (CPI). We show that the adaptive scaling mechanism used in TRPO is in fact the natural "RL version" of traditional trust-region methods from convex analysis. We first analyze TRPO in the planning setting, in which we have access to the model a…
In this paper we consider the use of probabilistic or random models within a classical trust-region framework for optimization of deterministic smooth general nonlinear functions. Our method and setting differs from many stochastic optimization approaches in two principal ways. Firstly, we assume that the value of the function itself can be computed without noise, in other words, that the function is deterministic. Secondly, we use random models of higher quality than those produced by usual stochastic gradient methods. In particular, a first order model based on random approximation of the gr…
The parallel solution of large scale non-linear programming problems, which arise for example from the discretization of non-linear partial differential equations, is a highly demanding task. Here, a novel solution strategy is presented, which is inherently parallel and globally convergent. Each global non-linear iteration step consists of asynchronous solutions of local non-linear programming problems followed by a global recombination step. The recombination step, which is the solution of a quadratic programming problem, is designed in a way such that it ensures global convergence. As it tur…
In this paper we develop efficient first-order algorithms for the generalized trust-region subproblem (GTRS), which has applications in signal processing, compressed sensing, and engineering. Although the GTRS, as stated, is nonlinear and nonconvex, it is well-known that objective value exactness holds for its SDP relaxation under a Slater condition. While polynomial-time SDP-based algorithms exist for the GTRS, their relatively large computational complexity has motivated and spurred the development of custom approaches for solving the GTRS. In particular, recent work in this direction has de…
In this paper, we study the local variational geometry of the optimal solution set of the trust region subproblem (TRS), which minimizes a general, possibly nonconvex, quadratic function over the unit ball. Specifically, we demonstrate that a Hölderian error bound holds globally for the TRS with modulus 1/4 and the Kurdyka-Łojasiewicz (KL) inequality holds locally for the TRS with a KL exponent 3/4 at any optimal solution. We further prove that unless in a special case, the Hölderian error bound modulus, as well as the KL exponent, is 1/2. Finally, based on the obtained KL property, we further…
ProxSTORM -- A Stochastic Trust-Region Algorithm for Nonsmooth Optimization
Sandia National Laboratories（桑迪亚国家实验室）；H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（乔治亚理工学院 H. 米尔顿·斯图尔特工业与系统工程学院）
The trust region subproblem (TRS) is to minimize a possibly nonconvex quadratic function over a Euclidean ball. There are typically two cases for (TRS), the so-called ``easy case'' and ``hard case''. Even in the ``easy case'', the sequence generated by the classical projected gradient method (PG) may converge to a saddle point at a sublinear local rate, when the initial point is arbitrarily selected from a nonzero measure feasible set. To our surprise, when applying (PG) to solve a cheap and possibly nonconvex reformulation of (TRS), the generated sequence initialized with {\it any} feasible p…
A stochastic second-order trust region method is proposed, which can be viewed as a second-order extension of the trust-region-ish (TRish) algorithm proposed by Curtis et al. (INFORMS J. Optim. 1(3) 200-220, 2019). In each iteration, a search direction is computed by (approximately) solving a trust region subproblem defined by stochastic gradient and Hessian estimates. The algorithm has convergence guarantees for stochastic minimization in the fully stochastic regime, meaning that guarantees hold when each stochastic gradient is required merely to be an unbiased estimate of the true gradient w…
We consider minimization of indefinite quadratics with either trust-region (norm) constraints or cubic regularization. Despite the nonconvexity of these problems we prove that, under mild assumptions, gradient descent converges to their global solutions, and give a non-asymptotic rate of convergence for the cubic variant. We also consider Krylov subspace solutions and establish sharp convergence guarantees to the solutions of both trust-region and cubic-regularized problems. Our rates mirror the behavior of these methods on convex quadratics and eigenvector problems, highlighting their scalabi…
Characterizing Trust and Resilience in Distributed Consensus for Cyberphysical Systems
Department of Electrical and Computer Engineering, Princeton University（普林斯顿大学电气工程与计算机科学系）；School of Electrical, Computer and Energy Engineering, Arizona State University（亚利桑那州立大学电气、计算机与能源工程学院）；Computer Science Department at the School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院计算机科学系）
