In this paper, we derive explicit second-order necessary and sufficient optimality conditions of a local minimizer to an optimal control problem for a quasilinear second-order partial differential equation with a piecewise smooth but not differentiable nonlinearity in the leading term. The key argument rests on the analysis of level sets of the state. Specifically, we show that if a function vanishes on the boundary and its the gradient is different from zero on a level set, then this set decomposes into finitely many closed simple curves. Moreover, the level sets depend continuously on the functions defining these sets. We also prove the continuity of the integrals on the level sets. In particular, Green's first identity is shown to be applicable on an open set determined by two functions with nonvanishing gradients. In the second part to this paper, the explicit sufficient second-order conditions will be used to derive error estimates for a finite-element discretization of the control problem.
In this paper, we present some new necessary and sufficient optimality conditions in terms of the Clarke subdifferentials for approximate Pareto solutions of a nonsmooth vector optimization problem which has an infinite number of constraints. As a consequence, we obtain optimality conditions for the particular cases of cone-constrained convex vector optimization problems and semidefinite vector optimization problems. Examples are given to illustrate the obtained results.
In this paper, we revisit the augmented Lagrangian method for a class of nonsmooth convex optimization. We present the Lagrange optimality system of the augmented Lagrangian associated with the problems, and establish its connections with the standard optimality condition and the saddle point condition of the augmented Lagrangian, which provides a powerful tool for developing numerical algorithms. We apply a linear Newton method to the Lagrange optimality system to obtain a novel algorithm applicable to a variety of nonsmooth convex optimization problems arising in practical applications. Unde…
In this paper we study an optimal control problem with nonsmooth mixed state and control constraints. In most of the existing results, the necessary optimality condition for optimal control problems with mixed state and control constraints are derived under the Mangasarian-Fromovitz condition and under the assumption that the state and control constraint functions are smooth. In this paper we derive necessary optimality conditions for problems with nonsmooth mixed state and control constraints under constraint qualifications based on pseudo-Lipschitz continuity and calmness of certain set-valu…
We employ a fuzzy optimality condition for the Frechet subdifferential and some advanced techniques of variational analysis such as formulae for the subdifferentials of an infinite family of nonsmooth functions and the coderivative scalarization to investigate robust optimality condition and robust duality for a nonsmooth/nonconvex multiobjective optimization problem dealing with uncertain constraints in arbitrary Asplund spaces. We establish necessary optimality conditions for weakly and properly robust efficient solutions of the problem in terms of the Mordukhovich subdifferentials of the re…
This paper considers a class of nonsmooth nonconvex-nonconcave min-max problems in machine learning and games. We first provide sufficient conditions for the existence of global minimax points and local minimax points. Next, we establish the first-order and second-order optimality conditions for local minimax points by using directional derivatives. These conditions reduce to smooth min-max problems with Fr{é}chet derivatives. We apply our theoretical results to generative adversarial networks (GANs) in which two neural networks contest with each other in a game. Examples are used to illustrat…
This paper deals with second-order optimality conditions for a quasilinear elliptic control problem with a nonlinear coefficient in the principal part that is countably $PC^2$ (continuous and $C^2$ apart from countably many points). We prove that the control-to-state operator is continuously differentiable even though the nonlinear coefficient is non-smooth. This enables us to establish "no-gap" second-order necessary and sufficient optimality conditions in terms of an abstract curvature functional, i. e., for which the sufficient condition only differs from the necessary one in the fact that …
The paper is devoted to an analysis of optimality conditions for nonsmooth multidimensional problems of the calculus of variations with various types of constraints, such as additional constraints at the boundary and isoperimetric constraints. To derive optimality conditions, we study generalised concepts of differentiability of nonsmooth functions called codifferentiability and quasidifferentiability. Under some natural and easily verifiable assumptions we prove that a nonsmooth integral functional defined on the Sobolev space is continuously codifferentiable and compute its codifferential an…
This paper deals with Pareto solutions of a nonsmooth fractional interval-valued multiobjective optimization. We first introduce four types of Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these solutions. Sufficient conditions for Pareto solutions of such a problem are also provided by means of introducing the concepts of (strictly) generalized convex functions defined in terms of the limiting/Mordukhovi…
In this article, we introduce the idea of $gH$-weak subdifferential for interval-valued functions (IVFs) and show how to calculate $gH$-weak subgradients. It is observed that a nonempty $gH$-weak subdifferential set is closed and convex. In characterizing the class of functions for which the $gH$-weak subdifferential set is nonempty, it is identified that this class is the collection of $gH$-lower Lipschitz IVFs. In checking the validity of sum rule of $gH$-weak subdifferential for a pair of IVFs, a counterexample is obtained, which reflects that the sum rule does not hold. However, under a mi…
In this article we propose a new approach to an analysis of DC optimization problems. This approach was largely inspired by codifferential calculus and the method of codifferential descent and is based on the use of a so-called affine support set of a convex function instead of the Frenchel conjugate function. With the use of affine support sets we define a global codifferential mapping of a DC function and derive new necessary and sufficient global optimality conditions for DC optimization problems. We also provide new simple necessary and sufficient conditions for the global exactness of the…
The paper is devoted to an analysis of a new constraint qualification and a derivation of the strongest existing optimality conditions for nonsmooth mathematical programming problems with equality and inequality constraints in terms of Demyanov-Rubinov-Polyakova quasidifferentials under the minimal possible assumptions. To this end, we obtain a novel description of convex subcones of the contingent cone to a set defined by quasidifferentiable equality and inequality constraints with the use of a new constraint qualification. We utilize these description and constraint qualification to derive t…
Optimality conditions in terms of Bouligand generalized differentials for a nonsmooth semilinear elliptic optimal control problem with distributed and boundary control pointwise constraints
This article is devoted to the analysis of necessary and/or sufficient conditions for metric regularity in terms of Demyanov-Rubinov-Polyakova quasidifferentials. We obtain new necessary and sufficient conditions for the local metric regularity of a multifunction in terms of quasidifferentials of the distance function to this multifunction. We also propose a new MFCQ-type constraint qualification for a parametric system of quasidifferentiable equality and inequality constraints and prove that it ensures the metric regularity of a multifunction associated with this system. As an application, we…
The main goal of this paper is developing the method of discrete approximations to derive necessary optimality conditions for a class of constrained sweeping processes with nonsmooth perturbations. Optimal control problems for sweeping processes have been recently recognized among the most interesting and challenging problems in modern control theory for discontinuous differential inclusions with irregular dynamics and implicit state constrained, while deriving necessary optimality conditions for their local minimizers have been significantly based on the smoothness of controlled dynamic pertu…
In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem. Moreover, it has since been argued that the asymptotic covariance of the method is best possible among any estimation procedure in a local minimax sense of Hájek and Le Cam. A long-standing open question in this line of work is whether similar guarantees hold for important non-smooth problems, such as stochastic nonlinear programming or stochastic variational inequalities. In this work, we show that this is indeed the case.
For a broad class of nonlinear systems, we formulate the problem of guaranteeing safety with optimality under constraints. Specifically, we define controlled safety for differential inclusions with constraints on the states and the inputs. Through the use of nonsmooth analysis tools, we show that a continuous optimal control law can be selected from a set-valued constraint capturing the system constraints and conditions guaranteeing safety using control barrier functions. Our results guarantee optimality and safety via a continuous state-feedback law designed using nonsmooth control barrier fu…
It is well known that there have been many numerical algorithms for solving nonsmooth minimax problems, numerical algorithms for nonsmooth minimax problems with joint linear constraints are very rare. This paper aims to discuss optimality conditions and develop practical numerical algorithms for minimax problems with joint linear constraints. First of all, we use the properties of proximal mapping and KKT system to establish optimality conditions. Secondly, we propose a framework of alternating coordinate algorithm for the minimax problem and analyze its convergence properties. Thirdly, we dev…
In this paper, we consider a class of optimal control problems for a one-dimensional time-discrete constrained quasilinear diffusion state-systems of singular Allen--Cahn types and its regularized approximating problems. We note that the control parameter for each system is given by physical temperature. The principal part of this paper is started with the verification of a Key-Theorem dealing with the decompositions of the subdifferentials of the governing convex energies of the state-systems. On this basis, we will prove five Main Theorems, concerned with: the solvability and precise regular…
We are interested in local quasi efficient solutions for nonsmooth vector optimization problems under new generalized approximate invexity assumptions. We formulate necessary and sufficient optimality conditions based on Stampacchia and Minty types of vector variational inequalities involving Clarke's generalized Jacobians. We also establish the relationship between local quasi weak efficient solutions and vector critical points.
The authors' paper in Optimization 63 (2014), 505-533, see Ref. [5], was the first one to provide detailed optimality conditions for pessimistic bilevel optimization. The results there were based on the concept of the two-level optimal value function introduced and analyzed in SIAM J. Optim. 22 (2012), 1309-1343; see Ref. [4], for the case of optimistic bilevel programs. One of the basic assumptions in both of these papers is that the functions involved in the problems are at least continuously differentiable. Motivated by the fact that many real-world applications of optimization involve func…
This paper focuses on the analysis of an optimal control problem governed by a nonsmooth quasilinear partial differential equation that models a stationary incompressible shear-thickening fluid. We start by studying the directional differentiability of the non-smooth term within the state equation as a prior step to demonstrate the directional differentiability of the solution operator. Thereafter, we establish a primal first order necessary optimality condition (Bouligand (B) stationarity), which is derived from the directional differentiability of the solution operator. By using a local regu…
We provide the first positive result on the nonsmooth optimization landscape of robust principal component analysis, to the best of our knowledge. It is the object of several conjectures and remains mostly uncharted territory. We identify a necessary and sufficient condition for the absence of spurious local minima in the rank-one case. Our proof exploits the subdifferential regularity of the objective function in order to eliminate the existence quantifier from the first-order optimality condition known as Fermat's rule.
These lecture notes for a graduate course cover generalized derivative concepts useful in deriving necessary optimality conditions and numerical algorithms for nondifferentiable optimization problems in inverse problems, imaging, and PDE-constrained optimization. Treated are convex functions and subdifferentials, Fenchel duality, monotone operators and resolvents, Moreau--Yosida regularization, proximal point and (some) first-order splitting methods, Clarke subdifferentials, and semismooth Newton methods. The required background from functional analysis and calculus of variations is also brief…
In this paper, we show that higher-order optimality conditions can be obtain for arbitrary nonsmooth function. We introduce a new higher-order directional derivative and higher-order subdifferential of Hadamard type of a given proper extended real function. This derivative is consistent with the classical higher-order Fréchet directional derivative in the sense that both derivatives of the same order coincide if the last one exists. We obtain necessary and sufficient conditions of order $n$ ($n$ is a positive integer) for a local minimum and isolated local minimum of order $n$ in terms of thes…
A general class of nonconvex optimization problems is considered, where the penalty is the composition of a linear operator with a nonsmooth nonconvex mapping, which is concave on the positive real line. The necessary optimality condition of a regularized version of the original problem is solved by means of a monotonically convergent scheme. Such problems arise in continuum mechanics, as for instance cohesive fractures, where singular behaviour is usually modelled by nonsmooth nonconvex energies. The proposed algorithm is successfully tested for fracture mechanics problems. Its performance is…
We propose a bilevel optimization strategy for selecting the best hyperparameter value for the nonsmooth $\ell_p$ regularizer with $0<p\le 1$. The concerned bilevel optimization problem has a nonsmooth, possibly nonconvex, $\ell_p$-regularized problem as the lower-level problem. Despite the recent popularity of nonconvex $\ell_p$-regularizer and the usefulness of bilevel optimization for selecting hyperparameters, algorithms for such bilevel problems have not been studied because of the difficulty of $\ell_p$-regularizer. Our contribution is the proposal of the first algorithm equipped with a …
We study the extension of the Chambolle--Pock primal-dual algorithm to nonsmooth optimization problems involving nonlinear operators between function spaces. Local convergence is shown under technical conditions including metric regularity of the corresponding primal-dual optimality conditions. We also show convergence for a Nesterov-type accelerated variant provided one part of the functional is strongly convex. We show the applicability of the accelerated algorithm to examples of inverse problems with $L^1$- and $L^\infty$-fitting terms as well as of state-constrained optimal control problem…
Learning to Control: The iUzawa-Net for Nonsmooth Optimal Control of Linear PDEs
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University（南洋理工大学数学科学系）；Department of Mathematics, The University of Hong Kong（香港大学数学系）；School of Mathematical Sciences, Nankai University（南开大学数学学院）
We propose a single time-scale stochastic subgradient method for constrained optimization of a composition of several nonsmooth and nonconvex functions. The functions are assumed to be locally Lipschitz and differentiable in a generalized sense. Only stochastic estimates of the values and generalized derivatives of the functions are used. The method is parameter-free. We prove convergence with probability one of the method, by associating with it a system of differential inclusions and devising a nondifferentiable Lyapunov function for this system. For problems with functions having Lipschitz …
We derive a variant of the nonsmooth maximum principle for problems with pure state constraints. The interest of our result resides on the nonsmoothness itself since, when applied to smooth problems, it coincides with known results. Remarkably, in the normal form, our result has the special feature of being a sufficient optimality condition for linearconvex problems, a feature that the classical Pontryagin maximum principle had whereas the nonsmooth version had not. This work is distinct to previous work in the literature since, for state constrained problems, we add the Weierstrass conditions…
In this paper, in terms of three types of generalized second-order derivatives of a nonsmooth function, we mainly study the corresponding second-order optimality conditions in a Hilbert space and prove the equivalence among these optimality conditions for paraconcave functions. As applications, we use these second-order optimality conditions to study strict local minimizers of order two and provide sufficient and/or necessary conditions for ensuring the local minimizer. This work extends and generalizes the study on second-order optimality conditions from the finite-dimensional space to the Hi…
We obtain a nonsmooth higher-order extension of Noether's symmetry theorem for variational isoperimetric problems with delayed arguments. The result is proved to be valid in the class of Lipschitz functions, as long as the delayed higher-order Euler--Lagrange extremals are restricted to those that satisfy the delayed higher-order DuBois--Reymond necessary optimality condition. The important case of delayed isoperimetric optimal control problems is considered as well.
We obtain a nonsmooth extension of Noether's symmetry theorem for variational problems with delayed arguments. The result is proved to be valid in the class of Lipschitz functions, as long as the delayed Euler-Lagrange extremals are restricted to those that satisfy the DuBois-Reymond necessary optimality condition. The important case of delayed variational problems with higher-order derivatives is considered as well.
This work is concerned with an optimal control problem governed by a non-smooth quasilinear elliptic equation with a nonlinear coefficient in the principal part that is locally Lipschitz continuous and directionally but not Gâteaux differentiable. This leads to a control-to-state operator that is directionally but not Gâteaux differentiable as well. Based on a suitable regularization scheme, we derive C- and strong stationarity conditions. Under the additional assumption that the nonlinearity is a PC^1 function with countably many points of nondifferentiability, we show that both conditions ar…
This paper deals with approximate solutions of a nonsmooth semi-infinite programming with multiple interval-valued objective functions. We first introduce four types of approximate quasi Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these approximate solutions. Sufficient conditions for approximate quasi Pareto solutions of such a problem are also provided by means of introducing the concepts of approxima…
In this article we utilise abstract convexity theory in order to unify and generalize many different concepts from nonsmooth analysis. We introduce the concepts of abstract codifferentiability, abstract quasidifferentiability and abstract convex (concave) approximations of a nonsmooth function mapping a topological vector space to an order complete topological vector lattice. We study basic properties of these notions, construct elaborate calculus of abstract codifferentiable functions and discuss continuity of abstract codifferential. We demonstrate that many classical concepts of nonsmooth a…
