In this paper, we tackle the long-standing challenges of ensemble control analysis and design using a convex-geometric approach in a Hilbert space setting. Specifically, we formulate the control of linear ensemble systems as a convex feasibility problem in a Hilbert space, which can be solved by iterative weighted projections. Such a non-trivial geometric interpretation not only enables a systematic design principle for constructing feasible, optimal, and constrained ensemble control signals, but also makes it possible for numerical examination of ensemble reachability and controllability. Furthermore, we incorporate this geometric approach into an iterative framework and illustrate its capability to derive feasible controls for steering bilinear ensemble systems. We conduct various numerical experiments on the control of linear and bilinear ensembles to validate the theoretical developments and demonstrate the applicability of the proposed convex-geometric approach.
Convergence of a projected stochastic gradient algorithm is demonstrated for convex objective functionals with convex constraint sets in Hilbert spaces. In the convex case, the sequence of iterates ${u_n}$ converges weakly to a point in the set of minimizers with probability one. In the strongly convex case, the sequence converges strongly to the unique optimum with probability one. An application to a class of PDE constrained problems with a convex objective, convex constraint and random elliptic PDE constraints is shown. Theoretical results are demonstrated numerically.
We establish finite convergence of circumcentered-reflection method (CRM) for the case of intersection of two closed convex cones in a real Hilbert space. We apply this result to prove the finite convergence for two polyhedral sets in R^n.
This work focuses on convergence analysis of the projected gradient method for solving constrained convex minimization problem in Hilbert spaces. We show that the sequence of points generated by the method employing the Armijo linesearch converges weakly to a solution of the considered convex optimization problem. Weak convergence is established by assuming convexity and Gateaux differentiability of the objective function, whose Gateaux derivative is supposed to be uniformly continuous on bounded sets. Furthermore, we propose some modifications in the classical projected gradient method in ord…
The Baillon-Haddad theorem establishes that the gradient of a convex and continuously differentiable function defined in a Hilbert space is $β$-Lipschitz if and only if it is $1/β$-cocoercive. In this paper, we extend this theorem to Gâteaux differentiable convex functions defined on an open convex set of a Hilbert space. Finally, we give a characterization of $C^{1,+}$ convex functions in terms of local cocoercitivity.
We prove minimax theorems for lower semicontinuous functions defined on a Hilbert space. The main tool is the theory of $Φ$-convex functions and sufficient and necessary conditions for the minimax equality to hold for $Φ$-convex functions. These conditions are expressed in terms of abstract $Φ$-subgradients.
In this work we focus on the convex feasibility problem (CFP) in Hilbert space. A specific method in this area that has gained a lot of interest in recent years is the Douglas-Rachford (DR) algorithm. This algorithm was originally introduced in 1956 for solving stationary and non-stationary heat equations. Then in 1979, Lions and Mercier adjusted and extended the algorithm with the aim of solving CFPs and even more general problems, such as finding zeros of the sum of two maximally monotone operators. Many developments which implement various concepts concerning this algorithm have occurred du…
The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the existence of, and necessary conditions for, optimal solutions of minimal angle problems associated with two convex subsets as well. Moreover, we generalize a result by Deutsch on minimal angles from lin…
We give a simple proof of the Baillon-Haddad theorem for convex functions defined on open and convex subsets of Hilbert spaces. We also state some generalizations and limitations. In particular, we discuss equivalent characterizations of the Lipschitz continuity of the derivative of convex functions on open and convex subsets of Banach spaces.
In this paper, we first study nonsmooth steepest descent method for nonsmooth functions defined on Hilbert space and establish the corresponding algorithm by proximal subgradients. Then, we use this algorithm to find stationary points for those functions satisfying prox-regularity and Lipschitzian continuity. As one application, the established algorithm is used to search the minimizer of lower semicontinuous and convex functions on finite-dimensional space. The convergent theorem, as one extension and improvement of the existing converging result for twice continuously differentiable convex f…
Convex feasibility problems require to find a point in the intersection of a finite family of convex sets. We propose to solve such problems by performing set-enlargements and applying a new kind of projection operators called valiant projectors. A valiant projector onto a convex set implements a special relaxation strategy, proposed by Goffin in 1971, that dictates the move toward the projection according to the distance from the set. Contrary to past realizations of this strategy, our valiant projection operator implements the strategy in a continuous fashion. We study properties of valiant …
The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^…
In this paper, a new optimization framework is defined that includes the optimization framework recently proposed in [1]-[2] as a special case. The convex optimization in [1]-[2] includes centralized optimization and distributed optimization over random networks, so does the optimization defined here. It is shown that the proposed algorithm in [2] converges almost surely and in mean square to a solution of the optimization problem here under suitable assumptions.
Stability of nonconvex quadratic programming problems under finitely many convex quadratic constraints in Hilbert spaces is investigated. We present several stability properties of the global solution map, and the continuity of the optimal value function, assuming that the problem data undergoes small perturbations.
In this paper, we consider the quadratic programming problems under finitely many convex quadratic constraints in Hilbert spaces. By using the Legendre property of quadratic forms or the compactness of operators in the presentations of quadratic forms, we establish some sufficient conditions for the solution existence of the considered problems. As special cases, we obtain some existence solution results for the quadratic programming problems under linear constraints in Hilbert spaces.
The flexibility of active ($p$) and reactive power ($q$) consumption in distributed energy resources (DERs) can be represented as a (potentially non-convex) set of points in the $p$-$q$ plane. Modeling of the aggregated flexibility in a heterogeneous ensemble of DERs as a Minkowski sum (M-sum) is computationally intractable even for moderately sized populations. In this article, we propose a scalable method of computing the M-sum of the flexibility domains of a heterogeneous ensemble of DERs, which are allowed to be non-convex, non-compact. In particular, the proposed algorithm computes a guar…
Purdue University（普渡大学）；University of Ioannina（约阿尼纳大学）
Given a {features, target} dataset, we introduce an incremental algorithm that constructs an aggregate regressor, using an ensemble of neural networks. It is well known that ensemble methods suffer from the multicollinearity issue, which is the manifestation of redundancy arising mainly due to the common training-dataset. In the present incremental approach, at each stage we optimally blend the aggregate regressor with a newly trained neural network under a convexity constraint which, if necessary, induces negative correlations. Under this framework, collinearity issues do not arise at all, re…
Because of Minty's classical correspondence between firmly nonexpansive mappings and maximally monotone operators, the notion of a firmly nonexpansive mapping has proven to be of basic importance in fixed point theory, monotone operator theory, and convex optimization. In this note, we show that if finitely many firmly nonexpansive mappings defined on a real Hilbert space are given and each of these mappings is asymptotically regular, which is equivalent to saying that they have or "almost have" fixed points, then the same is true for their composition. This significantly generalizes the resul…
A quadratically constrained quadratic programming problem is considered in a Hilbert space setting, where neither the objective nor the constraint are convex functions. Necessary and sufficient conditions are provided to guarantee that the problem admits solutions for every initial data (in an adequate set).
We develop a globalized Proximal Newton method for composite and possibly non-convex minimization problems in Hilbert spaces. Additionally, we impose less restrictive assumptions on the composite objective functional considering differentiability and convexity than in existing theory. As far as differentiability of the smooth part of the objective function is concerned, we introduce the notion of second order semi-smoothness and discuss why it constitutes an adequate framework for our Proximal Newton method. However, both global convergence as well as local acceleration still pertain to hold i…
Chebyshev Greedy Algorithm is a generalization of the well known Orthogonal Matching Pursuit defined in a Hilbert space to the case of Banach spaces. We apply this algorithm for constructing sparse approximate solutions (with respect to a given dictionary) to convex optimization problems. Rate of convergence results in a style of the Lebesgue-type inequalities are proved.
Strongly convex sets in Hilbert spaces are characterized by local properties. One quantity which is used for this purpose is a generalization of the modulus of convexity δ_Ωof a set Ω. We also show that \lim_{ε\to 0} δ_Ω(ε)/ε^2 exists whenever Ωis closed and convex.
