We address the problem of minimizing the long-run expected average cost of a complex system consisting of interactive subsystems. We formulate a multiobjective optimization problem of the one-stage expected costs of the subsystems and provide a duality framework to prove that the control policy yielding the Pareto optimal solution minimizes the average cost criterion of the system. We provide the conditions of existence and a geometric interpretation of the solution. For practical situations with constraints consistent to those studied here, our results imply that the Pareto control policy may be of value when we seek to derive online the optimal control policy in complex systems.
A dual control problem is presented for the optimal stochastic control of a system governed by partial differential equations. Relationships between the optimal values of the original and the dual problems are investigated and two duality theorems are proved. The dual problem serves to provide upper bounds for the optimal and maximum value of the original one or even to give the optimal value.
We associate with each convex optimization problem posed on some locally convex space with an infinite index set T, and a given non-empty family H formed by finite subsets of T, a suitable Lagrangian-Haar dual problem. We provide reverse H-strong duality theorems, H-Farkas type lemmas and optimality theorems. Special attention is addressed to infinite and semi-infinite linear optimization problems.
We consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is in duality with an optimization problem involving the Lagrange multiplier associated with the constraints. Then by convex analysis techniques, we provide a general existence result and some a priori estimation of the dual optimizers. We further provide a necessary and sufficient optimality condition for the initial constrained control problem. The same result…
This article studies convex duality in stochastic optimization over finite discrete-time. The first part of the paper gives general conditions that yield explicit expressions for the dual objective in many applications in operations research and mathematical finance. The second part derives optimality conditions by combining general saddle-point conditions from convex duality with the dual representations obtained in the first part of the paper. Several applications to stochastic optimization and mathematical finance are given.
A new formulation of Stochastic Model Predictive Output Feedback Control is presented and analyzed as a translation of Stochastic Optimal Output Feedback Control into a receding horizon setting. This requires lifting the design into a framework involving propagation of the conditional state density, the information state, via the Bayesian Filter and solution of the Stochastic Dynamic Programming Equation for an optimal feedback policy, both stages of which are computationally challenging in the general, nonlinear setup. The upside is that the clearance of three bottleneck aspects of Model Pred…
This paper studies duality and optimality conditions in general convex stochastic optimization problems introduced by Rockafellar and Wets in 1976. We derive an explicit dual problem in terms of two dual variables, one of which is the shadow price of information while the other one gives the marginal cost of a perturbation much like in classical Lagrangian duality. Existence of primal solutions and the absence of duality gap are obtained without compactness or boundedness assumptions. In the context of financial mathematics, the relaxed assumptions are satisfied under the well-known no-arbitra…
The paper studies optimal control problem described by higher order evolution differential inclusions (DFIs) with endpoint and state constraints. In the term of Euler-Lagrange type inclusion is derived sufficient condition of optimality for higher order DFIs. It is shown that the adjoint inclusion for the first order DFIs, defined in terms of locally adjoint mapping, coincides with the classical Euler-Lagrange inclusion. Then a duality theorem is proved, which shows that Euler-Lagrange inclusions are "duality relations" for both problems. At the end of the paper duality problems for third orde…
This paper proposes a general duality framework for the problem of minimizing a convex integral functional over a space of stochastic processes adapted to a given filtration. The framework unifies many well-known duality frameworks from operations research and mathematical finance. The unification allows the extension of some useful techniques from these two fields to a much wider class of problems. In particular, combining certain finite-dimensional techniques from convex analysis with measure theoretic techniques from mathematical finance, we are able to close the duality gap in some situati…
Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs in short) are formulated and studied. A general duality principle is established for linear backward stochastic integral equation and linear stochastic Fredholm-Volterra integral equation with mean-field. With the help of such a duality principle, together with some other new delicate and subtle skills, Pontryagin type maximum principles are proved for two optimal control problems of FBSVIEs.
A stochastic linear quadratic (LQ) optimal control problem with a pointwise linear equality constraint on the terminal state is considered. A strong Lagrangian duality theorem is proved under a uniform convexity condition on the cost functional and a surjectivity condition on the linear constraint mapping. Based on the Lagrangian duality, two approaches are proposed to solve the constrained stochastic LQ problem. First, a theoretical method is given to construct the closed-form solution by the strong duality. Second, an iterative algorithm, called augmented Lagrangian method (ALM), is proposed…
We use the technique of information relaxation to develop a duality-driven iterative approach to obtaining and improving confidence interval estimates for the true value of finite-horizon stochastic dynamic programming problems. We show that the sequence of dual value estimates yielded from the proposed approach in principle monotonically converges to the true value function in a finite number of dual iterations. Aiming to overcome the curse of dimensionality in various applications, we also introduce a regression-based Monte Carlo algorithm for implementation. The new approach can be used not…
This paper proposes a set of closed-form conditions to ensure the strong duality of second-order cone program (SOCP) formulation for AC power flow in radial power networks. In addition, numerical evaluations on IEEE 33-bus test networks and a real-world distribution network are performed to demonstrate the validity of the proposed conditions.
We develop a general theory of convex duality for certain singular control problems, taking the abstract results by Kramkov and Schachermayer (1999) for optimal expected utility from nonnegative random variables to the level of optimal expected utility from increasing, adapted controls. The main contributions are the formulation of a suitable duality framework, the identification of the problem's dual functional as well as the full duality for the primal and dual value functions and their optimizers. The scope of our results is illustrated by an irreversible investment problem and the Hindy-Hu…
This paper mainly addresses the Monge mass transfer problem in the 1-D case. Through an ingenious approximation mechanism, one transforms the Monge problem into a sequence of minimization problems, which can be converted into a sequence of nonlinear differential equations with constraints by variational method. The existence and uniqueness of the solution for each equation can be demonstrated by applying the canonical duality method. Moreover, the duality method gives a sequence of perfect dual maximization problems. In the final analysis, one constructs the approximation of optimal mapping fo…
This paper deals with Pareto solutions of a nonsmooth fractional interval-valued multiobjective optimization. We first introduce four types of Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these solutions. Sufficient conditions for Pareto solutions of such a problem are also provided by means of introducing the concepts of (strictly) generalized convex functions defined in terms of the limiting/Mordukhovi…
We consider a Markov chain approximation scheme for utility maximization problems in continuous time, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite approximation of the Gaussian increments. The error estimates previously derived in Picarelli and Reisinger (2019) are asymmetric between lower and upper bounds due to the control approximation and improve on known results in the literature in the lower case only. In the present paper, we use duality results to obtain a posteriori upper error bounds which are empirically of the sam…
One of the most important optimality conditions to aid to solve a vector optimization problem is the first-order necessary optimality condition that generalizes the Karush-Kuhn-Tucker condition. However, to obtain the sufficient optimality conditions, it is necessary to impose additional assumptions on the objective functions and in the constraint set. The present work is concerned with the constrained vector quadratic fractional optimization problem. It shows that sufficient Pareto optimality conditions and the main duality theorems can be established without the assumption of generalized con…
The paper deals with the optimal control problem described by second order evolution differential inclusions; to this end first we use an auxiliary problem with second order discrete and discrete-approximate inclusions. Then applying infimal convolution concept of convex functions, step by step we construct the dual problems for discrete, discrete-approximate and differential inclusions and prove duality results. It seems that the Euler-Lagrange type inclusions are "duality relations" for both primary and dual problems and that the dual problem for discrete-approximate problem make a bridge be…
MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Inria（法国国家信息与自动化研究所）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL Research University（巴黎文理研究大学）
We consider stochastic gradient methods under the interpolation regime where a perfect fit can be obtained (minimum loss at each observation). While previous work highlighted the implicit regularization of such algorithms, we consider an explicit regularization framework as a minimum Bregman divergence convex feasibility problem. Using convex duality, we propose randomized Dykstra-style algorithms based on randomized dual coordinate ascent. For non-accelerated coordinate descent, we obtain an algorithm which bears strong similarities with (non-averaged) stochastic mirror descent on specific fu…
This paper on the whole concerns with the duality of Mayer problem for k-th order differential inclusions, where k is an arbitrary natural number. Thus, this work for constructing the dual problems to differential inclusions of any order can make a great contribution to the modern development of optimal control theory. To this end in the form of Euler-Lagrange type inclusions and transversality conditions the sufficient optimality conditions are derived. The principal idea of obtaining optimal conditions is locally adjoint mappings. It appears that the Euler-Lagrange type inclusions for both p…
This paper introduces a dual problem to study a continuous-time consumption and investment problem with incomplete markets and stochastic differential utility. For Epstein-Zin utility, duality between the primal and dual problems is established. Consequently the optimal strategy of the consumption and investment problem is identified without assuming several technical conditions on market model, utility specification, and agent's admissible strategy. Meanwhile the minimizer of the dual problem is identified as the utility gradient of the primal value and is economically interpreted as the "lea…
We consider the Monge-Kantorovich transport problem in an abstract measure theoretic setting. Our main result states that duality holds if $c:X\times Y\to [0,\infty)$ is an arbitrary Borel measurable cost function on the product of Polish spaces $X,Y$. In the course of the proof we show how to relate a non - optimal transport plan to the optimal transport costs via a ``subsidy'' function and how to identify the dual optimizer. We also provide some examples showing the limitations of the duality relations.
This paper is concerned with the optimal control problem governed by a linear parabolic equation and subjected to box constraints on control variables. This type of problem has important applications in heating and cooling systems. By applying the scheme of Fenchel duality, we derive the dual problem explicitly where the control constraints in primal problem are embedded in the dual problem's objective functional. The existence and uniqueness of the solution to the dual problem are proved and the first-order optimality conditions are also derived. In addition, we discuss the saddle point prope…
For a primal-dual pair of conic linear problems that are described by convex cones $S\subset X$, $T\subset Y$, bilinear symmetric objective functions $\langle\cdot,\cdot\rangle_X$, $\langle\cdot,\cdot\rangle_Y$ and a linear operator $A:X\rightarrow Y$, we show that the existence of optimal solutions $x^*\in S$, $y^*\in T$ that satisfy $Ax^*=b$ and $A^Ty^*=c$ eventually comes down to the consistency and solvability of the problems $min\langle z,z\rangle_Y,\;z\in\{Ax-b:x\in S\}$ and $ min\langle w,w\rangle_X,\; w\in\{A^Ty-c:y\in T\}$. Assuming that these two problems are consistent and solvable,…
In this paper, we present a convex formulation of $H_{\infty}$-optimal control problem for coupled linear ODE-PDE systems with one spatial dimension. First, we reformulate the coupled ODE-PDE system as a Partial Integral Equation (PIE) system and show that stability and $H_{\infty}$ performance of the PIE system implies that of the ODE-PDE system. We then construct a dual PIE system and show that asymptotic stability and $H_{\infty}$ performance of the dual system is equivalent to that of the primal PIE system. Next, we pose a convex dual formulation of the stability and $H_{\infty}$-performan…
In this paper, we consider a special class of nonlinear optimal control problems, where the control variables are box-constrained and the objective functional is strongly convex corresponding to control variables and separable with respect to the state variables and control variables. We convert solving the original nonlinear problem into solving a sequence of constrained linear-quadratic optimal control problems by quasilinearization method. In order to solve each linear-quadratic problem efficiently we turn to study its dual problem. We formulate dual problem by the scheme of Fenchel duality…
Duality of control and estimation allows mapping recent advances in data-guided control to the estimation setup. This paper formalizes and utilizes such a mapping to consider learning the optimal (steady-state) Kalman gain when process and measurement noise statistics are unknown. Specifically, building on the duality between synthesizing optimal control and estimation gains, the filter design problem is formalized as direct policy learning. In this direction, the duality is used to extend existing theoretical guarantees of direct policy updates for Linear Quadratic Regulator (LQR) to establis…
Motivated by energy management for micro-grids, we study convex optimization problems with uncertainty in the objective function and sequential decision making. To solve these problems, we propose a new framework called ``Online Duality-Driven Optimization'' (ODDO). This framework distinguishes itself from existing paradigms for optimization under uncertainty in its efficiency, simplicity, and ability to solve problems without any quantitative assumptions on the uncertain data. The key idea in this framework is that we predict, instead of the actual uncertain data, the optimal Lagrange multipl…
This paper is concerned with the development and use of duality theory for a nonlinear filtering model with white noise observations. The main contribution of this paper is to introduce a stochastic optimal control problem as a dual to the nonlinear filtering problem. The mathematical statement of the dual relationship between the two problems is given in the form of a duality principle. The constraint for the optimal control problem is the backward stochastic differential equation (BSDE) introduced in the companion paper. The optimal control solution is obtained from an application of the max…
We revisit the foundations of gauge duality and demonstrate that it can be explained using a modern approach to duality based on a perturbation framework. We therefore put gauge duality and Fenchel-Rockafellar duality on equal footing, including explaining gauge dual variables as sensitivity measures, and showing how to recover primal solutions from those of the gauge dual. This vantage point allows a direct proof that optimal solutions of the Fenchel-Rockafellar dual of the gauge dual are precisely the primal solutions rescaled by the optimal value. We extend the gauge duality framework to th…
This paper studies convex problems of Bolza in the conjugate duality framework of Rockafellar. We parameterize the problem by a general Borel measure which has direct economic interpretation in problems of financial economics. We derive a dual representation for the optimal value function in terms of continuous dual arcs and we give conditions for the existence of solutions. Combined with well-known results on problems of Bolza over absolutely continuous arcs, we obtain optimality conditions in terms of extended Hamiltonian conditions.
We extend in two ways the standard Karush-Kuhn-Tucker optimality conditions to problems with a convex objective, convex functional constraints, and the extra requirement that some of the variables must be integral. While the standard Karush-Kuhn-Tucker conditions involve separating hyperplanes, our extension is based on lattice-free polyhedra. Our optimality conditions allow us to define an exact dual of our original mixed-integer convex problem.
Using the duality techniques introduced by De Meyer (1996a, 1996b), De Meyer and Marino (2005) provided optimal strategies for both players in finitely repeated games with incomplete information on two sides, in the independent case. In this note, we extend both the duality techniques and the construction of optimal strategies to the case of general type dependence.
SIT, FEIT, The University of Sydney（悉尼大学工程与信息技术学院）；University of Rochester（罗切斯特大学）
We consider the composition optimization with two expected-value functions in the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement learning and nonlinear embedding}. Full Gradient or classical stochastic gradient descent based optimization algorithms are unsuitable or computationally expensive to solve this problem due to the inner expectation $\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x)$. We propose a duality-…
We consider Continuous Linear Programs over a continuous finite time horizon $T$, with linear cost coefficient functions and linear right hand side functions and a constant coefficient matrix, where we search for optimal solutions in the space of measures or of functions of bounded variation. These models generalize the Separated Continuous Linear Programming models and their various duals, as formulated in the past by Anderson, by Pullan, and by Weiss. We present simple necessary and sufficient conditions for feasibility. We formulate a symmetric dual and investigate strong duality by conside…
In a recent paper it has been shown that if Cesaro and Abel limits for a certain discrete time optimal control problem are not equal, then there is a duality gap between a certain infinite-dimensional linear programming problem and its dual. In this paper we construct an example of a problem satisfying the assumptions of the aforementioned paper, where Cesaro and Abel limits are different.
We revisit the duality theorem for multimarginal optimal transportation problems. In particular, we focus on the Coulomb cost. We use a discrete approximation to prove equality of the extremal values and some careful estimates of the approximating sequence to prove existence of maximizers for the dual problem (Kantorovich's potentials). Finally we observe that the same strategy can be applied to a more general class of costs and that a classical results on the topic cannot be applied here.
We study the optimal transport between two probability measures on the real line, where the transport plans are laws of one-step martingales. A quasi-sure formulation of the dual problem is introduced and shown to yield a complete duality theory for general marginals and measurable reward (cost) functions: absence of a duality gap and existence of dual optimizers. Both properties are shown to fail in the classical formulation. As a consequence of the duality result, we obtain a general principle of cyclical monotonicity describing the geometry of optimal transports.
Optimal transportation with capacity constraints, a variant of the well-known optimal transportation problem, is concerned with transporting one probability density $f \in L^1(\mathbb{R}^m)$ onto another one $g \in L^1(\mathbb{R}^n)$ so as to optimize a cost function $c \in L^1(\mathbb{R}^{m+n})$ while respecting the capacity constraints $0\le h \le \bar h\in L^\infty(\mathbb{R}^{m+n})$. A linear programming duality theorem for this problem was first established by Levin. In this note, we prove under mild assumptions on the given data, the existence of a pair of $L^1$-functions optimizing the …
Duality between estimation and optimal control is a problem of rich historical significance. The first duality principle appears in the seminal paper of Kalman-Bucy, where the problem of minimum variance estimation is shown to be dual to a linear quadratic (LQ) optimal control problem. Duality offers a constructive proof technique to derive the Kalman filter equation from the optimal control solution. This paper generalizes the classical duality result of Kalman-Bucy to the nonlinear filter: The state evolves as a continuous-time Markov process and the observation is a nonlinear function of st…
