We consider in this paper a class of single-ratio fractional minimization problems, in which the numerator part of the objective is the sum of a nonsmooth nonconvex function and a smooth nonconvex function while the denominator part is a nonsmooth convex function. In this work, we first derive its first-order necessary optimality condition, by using the first-order operators of the three functions involved. Then we develop first-order algorithms, namely, the proximity-gradient-subgradient algorithm (PGSA), PGSA with monotone line search (PGSA_ML) and PGSA with nonmonotone line search (PGSA_NL). It is shown that any accumulation point of the sequence generated by them is a critical point of the problem under mild assumptions. Moreover, we establish global convergence of the sequence generated by PGSA or PGSA_ML and analyze its convergence rate, by further assuming the local Lipschitz continuity of the nonsmooth function in the numerator part, the smoothness of the denominator part and the Kurdyka- Lojasiewicz property of the objective. The proposed algorithms are applied to the sparse generalized eigenvalue problem associated with a pair of symmetric positive semidefinite matrices and the corresponding convergence results are obtained according to their general convergence theorems. We perform some preliminary numerical experiments to demonstrate the efficiency of the proposed algorithms
In this paper, we employ a reduced basis method for solving the PDE constrained optimization problem governed by a fractional parabolic equation with the fractional derivative in time from order beta in (0,1) is defined by Caputo fractional derivative.
Based on a method introduced by Leitmann [Internat. J. Non-Linear Mech. {\bf 2} (1967), 55--59], we exhibit exact solutions for some fractional optimization problems of the calculus of variations and optimal control.
This paper deals with Pareto solutions of a nonsmooth fractional interval-valued multiobjective optimization. We first introduce four types of Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these solutions. Sufficient conditions for Pareto solutions of such a problem are also provided by means of introducing the concepts of (strictly) generalized convex functions defined in terms of the limiting/Mordukhovi…
A method is given for finding roots of a one-variable function using Taylor's expansion of that function and fractional derivative calculated at a suitable tangent point without using Newton's method, but is regarded as a variant of Halley and Newton's one. Several examples regarding polynomials are stated as well. Then, the given method is generalized to functions of several variables belonging to an $n$-dimensional space and one example is given for optimization and solution of a nonlinear system of equations by both our method and Gradient Descent one. A comparison of our method is made wit…
We extend Robust Optimization to fractional programming, where both the objective and the constraints contain uncertain parameters. Earlier work did not consider uncertainty in both the objective and the constraints, or did not use Robust Optimization. Our contribution is threefold. First, we provide conditions to guarantee that either a globally optimal solution, or a sequence converging to the globally optimal solution, can be found by solving one or more convex optimization problems. Second, we identify two cases for which an exact solution can be obtained by solving a single optimization p…
Motivated by gradient methods in optimization theory, we give methods based on $ψ$-fractional derivatives of order $α$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving $ψ$-fractional derivatives. Numerical examples using the ABM method show that the fractional order $α$ and weight $ψ$ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.
One of the most important optimality conditions to aid to solve a vector optimization problem is the first-order necessary optimality condition that generalizes the Karush-Kuhn-Tucker condition. However, to obtain the sufficient optimality conditions, it is necessary to impose additional assumptions on the objective functions and in the constraint set. The present work is concerned with the constrained vector quadratic fractional optimization problem. It shows that sufficient Pareto optimality conditions and the main duality theorems can be established without the assumption of generalized con…
In this note we study multiple-ratio fractional 0--1 programs, a broad class of NP-hard combinatorial optimization problems. In particular, under some relatively mild assumptions we provide a complete characterization of the conditions, which ensure that a single-ratio function is submodular. Then we illustrate our theoretical results with the assortment optimization and facility location problems, and discuss practical situations that guarantee submodularity in the considered application settings. In such cases, near-optimal solutions for multiple-ratio fractional 0--1 programs can be found v…
We develop a simple and accurate method to solve fractional variational and fractional optimal control problems with dependence on Caputo and Riemann-Liouville operators. Using known formulas for computing fractional derivatives of polynomials, we rewrite the fractional functional dynamical optimization problem as a classical static optimization problem. The method for classical optimal control problems is called Ritz's method. Examples show that the proposed approach is more accurate than recent methods available in the literature.
There are several concepts and definitions that characterize and give optimality conditions for solutions of a vector optimization problem. One of the most important is the first-order necessary optimality condition that generalizes the Karush-Kuhn-Tucker condition. This condition ensures the existence of an arbitrary neighborhood that contains an local optimal solution. The present work we introduce an alternative concept to identify the local optimal solution neighborhood in vector optimization problems. The main aspect of this contribution is the development of necessary and sufficient Pare…
Improperly efficient solutions in the sense of Geoffrion in linear fractional vector optimization problems with unbounded constraint sets are studied in this paper. We give two sets of conditions which assure that all the efficient solutions of a given problem are improperly efficient. We also obtain necessary conditions for an efficient solution to be improperly efficient. As a result, we have new sufficient conditions for Geoffrion's proper efficiency. The obtained results enrich our knowledge on properly efficient solutions in linear fractional vector optimization.
In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. …
The paper is devoted to the study of the unconditional extremal problem for a fractional linear integral functional defined on a set of probability distributions. In contrast to results proved earlier, the integrands of the integral expressions in the numerator and the denominator in the problem under consideration depend on a real optimization parameter vector. Thus, the optimization problem is studied on the Cartesian product of a set of probability distributions and a set of admissible values of a real parameter vector. Three statements on the extremum of a fractional linear integral functi…
Integration by parts plays a crucial role in mathematical analysis, e.g., during the proof of necessary optimality conditions in the calculus of variations and optimal control. Motivated by this fact, we construct a new, right-weighted generalized fractional derivative in the Riemann-Liouville sense with its associated integral for the recently introduced weighted generalized fractional derivative with Mittag-Leffler kernel. We rewrite these operators equivalently in effective series, proving some interesting properties relating to the left and the right fractional operators. These results per…
Nonlocal operators of fractional type are a popular modeling choice for applications that do not adhere to classical diffusive behavior; however, one major challenge in nonlocal simulations is the selection of model parameters. In this work we propose an optimization-based approach to parameter identification for fractional models with an optional truncation radius. We formulate the inference problem as an optimal control problem where the objective is to minimize the discrepancy between observed data and an approximate solution of the model, and the control variables are the fractional order …
George Mason University（乔治梅森大学）；Argonne National Laboratory（阿贡国家实验室）
In this work we consider a generalized bilevel optimization framework for solving inverse problems. We introduce fractional Laplacian as a regularizer to improve the reconstruction quality, and compare it with the total variation regularization. We emphasize that the key advantage of using fractional Laplacian as a regularizer is that it leads to a linear operator, as opposed to the total variation regularization which results in a nonlinear degenerate operator. Inspired by residual neural networks, to learn the optimal strength of regularization and the exponent of fractional Laplacian, we de…
This paper introduces a novel algorithmic framework for a deep neural network (DNN), which in a mathematically rigorous manner, allows us to incorporate history (or memory) into the network -- it ensures all layers are connected to one another. This DNN, called Fractional-DNN, can be viewed as a time-discretization of a fractional in time nonlinear ordinary differential equation (ODE). The learning problem then is a minimization problem subject to that fractional ODE as constraints. We emphasize that an analogy between the existing DNN and ODEs, with standard time derivative, is well-known by …
In this note we present a multigrid preconditioning method for solving quadratic optimization problems constrained by a fractional diffusion equation. Multigrid methods within the all-at-once approach to solve the first order-order optimality Karush-Kuhn-Tucker (KKT) systems are widely popular, but their development have relied on the underlying systems being sparse. On the other hand, for most discretizations, the matrix representation of fractional operators is expected to be dense. We develop a preconditioning strategy for our problem based on a reduced approach, namely we eliminate the sta…
The aim of this study is to find the optimum of a linear fractional function over the efficient set of a multi-objective linear fractional integer program without generating all efficient solutions. By its nature, it is a global optimization problem since the efficient set is discrete, hence not convex. For this purpose, a branch and bound based method is described with a double mission to search for an optimal solution for a given linear fractional function which is moreover, efficient for a multi-objective linear fractional integer programming problem. Tests performed on instances randomly g…
Distributed-order fractional non-local operators have been introduced and studied by Caputo at the end of the 20th century. They generalize fractional order derivatives/integrals in the sense that such operators are defined by a weighted integral of different orders of differentiation over a certain range. The subject of distributed-order non-local derivatives is currently under strong development due to its applications in modeling some complex real world phenomena. Fractional optimal control theory deals with the optimization of a performance index functional subject to a fractional control …
Linear programming (LP) relaxation is a standard technique for solving hard combinatorial optimization (CO) problems. Here we present a gradient descent algorithm which exploits the special structure of some LP relaxations induced by CO problems. The algorithm can be run in parallel mode and was implemented as CUDA C/C++ program to be executed on GPU. We exemplify efficiency of the algorithm by solving a fractional 2-matching problem. Our results demonstrate that a fractional 2-matching problem with 100,000 nodes is solved by our algorithm on a modern GPU on a scale of a second while solving t…
In this paper, we study a class of fractional optimal control problems. A necessary condition for the existence of an optimal control is provided in the literature. It is commonly given as the existence of a solution of a fractional Pontryagin's system and the proof is based on the introduction of a Lagrange multiplier. Assuming an additional condition on these problems, we suggest a new presentation of this result with a proof using only classical mathematical tools adapted to the fractional case: calculus of variations, Gronwall's Lemma, Cauchy-Lipschitz Theorem and stability under perturbat…
In this paper, we propose an energy-efficient optimal altitude for an aerial access point (AAP), which acts as a flying base station to serve a set of ground user equipment (UE). Since the ratio of total energy consumed by the aerial vehicle to the communication energy is very large, we include the aerial vehicle's energy consumption in the problem formulation. After considering the energy consumption model of the aerial vehicle, our objective is translated into a non-convex optimization problem of maximizing the global energy efficiency (GEE) of the aerial communication system, subject to alt…
In this paper, we consider a broad class of nonsmooth and nonconvex fractional programs, where the numerator can be written as the sum of a continuously differentiable convex function whose gradient is Lipschitz continuous and a proper lower semicontinuous (possibly nonconvex) function, and the denominator is weakly convex over the constraint set. This model problem includes the composite optimization problems studied extensively lately, and encompasses many important modern fractional optimization problems arising from diverse areas such as the recently proposed scale invariant sparse signal …
This paper presents a canonical dual approach to the problem of minimizing the sum of a quadratic function and the ratio of nonconvex function and quadratic functions, which is a type of non-convex optimization problem subject to an elliptic constraint. We first relax the fractional structure by introducing a family of parametric subproblems. Under certain conditions, we show that the canonical dual of each subproblem becomes a two-dimensional concave maximization problem that exhibits no duality gap. Since the infimum of the optima of the parameterized subproblems leads to a solution to the o…
We study operators that are generalizations of the classical Riemann-Liouville fractional integral, and of the Riemann-Liouville and Caputo fractional derivatives. A useful formula relating the generalized fractional derivatives is proved, as well as three relations of fractional integration by parts that change the parameter set of the given operator into its dual. Such results are explored in the context of dynamic optimization, by considering problems of the calculus of variations with general fractional operators. Necessary optimality conditions of Euler-Lagrange type and natural boundary …
Fractional operators play an important role in modeling nonlocal phenomena and problems involving coarse-grained and fractal spaces. The fractional calculus of variations with functionals depending on derivatives and/or integrals of noninteger order is a rather recent subject that is currently in fast development due to its applications in physics and other sciences. In the last decade, several approaches to fractional variational calculus were proposed by using different notions of fractional derivatives and integrals. Although the literature of the fractional calculus of variations is alread…
We propose a novel Caputo fractional derivative-based optimization algorithm. Upon defining the Caputo fractional gradient with respect to the Cartesian coordinate, we present a generic Caputo fractional gradient descent (CFGD) method. We prove that the CFGD yields the steepest descent direction of a locally smoothed objective function. The generic CFGD requires three parameters to be specified, and a choice of the parameters yields a version of CFGD. We propose three versions -- non-adaptive, adaptive terminal and adaptive order. By focusing on quadratic objective functions, we provide a conv…
Recent studies on diffusion-based sampling methods have shown that Langevin Monte Carlo (LMC) algorithms can be beneficial for non-convex optimization, and rigorous theoretical guarantees have been proven for both asymptotic and finite-time regimes. Algorithmically, LMC-based algorithms resemble the well-known gradient descent (GD) algorithm, where the GD recursion is perturbed by an additive Gaussian noise whose variance has a particular form. Fractional Langevin Monte Carlo (FLMC) is a recently proposed extension of LMC, where the Gaussian noise is replaced by a heavy-tailed α-stable noise. …
In this paper, we utilize ADCSO (Adaptive Dynamic Cat Swarm Optimization) to estimate the parameters of Fractional Order Grey Model. The parameters of Fractional Order Grey Model affect the prediction accuracy of the model. In order to solve the problem that general swarm intelligence algorithms easily fall into the local optimum and optimize the accuracy of the model, ADCSO is utilized to reduce the error of the model. Experimental results for the data of container throughput of Wuhan Port and marine capture productions of Zhejiang Province show that the different parameter values affect the …
Distributed Fractional Bayesian Learning for Adaptive Optimization
Department of Control Science and Engineering, Tongji University（控制科学与工程系，同济大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）；Frontiers Science Center for Intelligent Autonomous Systems, Ministry of Education（智能自主系统前沿科学中心，教育部）；Shanghai Institute of Intelligent Science and Technology, Tongji University（上海智能科学与技术研究院，同济大学）
University of Washington（华盛顿大学）；McGill University（麦吉尔大学）
Generalized matrix-fractional (GMF) functions are a class of matrix support functions introduced by Burke and Hoheisel as a tool for unifying a range of seemingly divergent matrix optimization problems associated with inverse problems, regularization and learning. In this paper we dramatically simplify the support function representation for GMF functions as well as the representation of their subdifferentials. These new representations allow the ready computation of a range of important related geometric objects whose formulations were previously unavailable.
Saarland University（萨尔大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
The (constrained) minimization of a ratio of set functions is a problem frequently occurring in clustering and community detection. As these optimization problems are typically NP-hard, one uses convex or spectral relaxations in practice. While these relaxations can be solved globally optimally, they are often too loose and thus lead to results far away from the optimum. In this paper we show that every constrained minimization problem of a ratio of non-negative set functions allows a tight relaxation into an unconstrained continuous optimization problem. This result leads to a flexible framew…
