In [13], an Inexact variant of Stochastic Dual Dynamic Programming (SDDP) called ISDDP was introduced which uses approximate (instead of exact with SDDP) primal dual solutions of the problems solved in the forward and backward passes of the method. That variant of SDDP was studied in [13] for linear and for differentiable nonlinear Multistage Stochastic Programs (MSPs). In this paper, we extend ISDDP to nondifferentiable MSPs. We first provide formulas for inexact cuts for value functions of convex nondifferentiable optimization problems. We then combine these cuts with SDDP to describe ISDDP for nondifferentiable MSPs and analyze the convergence of the method. More precisely, for a problem with T stages, we show that for errors bounded from above by epsilon, the limit superior and limit inferior of sequences of upper and lower bounds on the optimal value of the problem are at most at distance 3*epsilon*T to the optimal value and that for asymptotically vanishing errors ISDDP converges to an optimal policy.
[13] V. Guigues, Inexact cuts in Stochastic Dual Dynamic Programming, Siam Journal on Optimization, 30(1), 407-438, 2020.
We introduce a variant of Multicut Decomposition Algorithms (MuDA), called CuSMuDA (Cut Selection for Multicut Decomposition Algorithms), for solving multistage stochastic linear programs that incorporates a class of cut selection strategies to choose the most relevant cuts of the approximate recourse functions. This class contains Level 1 and Limited Memory Level 1 cut selection strategies, initially introduced for respectively Stochastic Dual Dynamic Programming (SDDP) and Dual Dynamic Programming (DDP). We prove the almost sure convergence of the method in a finite number of iterations and …
In this paper, we extend the adaptive partition-based approach for solving two-stage stochastic programs with fixed recourse to the multistage stochastic programming setting. The proposed algorithms integrate the adaptive partition-based strategy with a popular approach for solving multistage stochastic programs, the stochastic dual dynamic programming, via different tree-traversal strategies in order to enhance its computational efficiency. Our numerical experiments on a hydro-thermal power generation planning problem show the effectiveness of the proposed algorithms.
We investigate the dual of a Multistage Stochastic Linear Program (MSLP) to study two questions for this class of problems. The first of these questions is the study of the optimal value of the problem as a function of the involved parameters. For this sensitivity analysis problem, we provide formulas for the derivatives of the value function with respect to the parameters and illustrate their application on an inventory problem. Since these formulas involve optimal dual solutions, we need an algorithm that computes such solutions to use them. In this context, the second question we address is…
We introduce StoDCuP (Stochastic Dynamic Cutting Plane), an extension of the Stochastic Dual Dynamic Programming (SDDP) algorithm to solve multistage stochastic convex optimization problems. At each iteration, the algorithm builds lower affine functions not only for the cost-to-go functions, as SDDP does, but also for some or all nonlinear cost and constraint functions. We show the almost sure convergence of StoDCuP. We also introduce an inexact variant of StoDCuP where all subproblems are solved approximately (with bounded errors) and show the almost sure convergence of this variant for vanis…
There has been widespread interest in the use of grid-level storage to handle the variability from increasing penetrations of wind and solar energy. This problem setting requires optimizing energy storage and release decisions for anywhere from a half-dozen, to potentially hundreds of storage devices spread around the grid as new technologies evolve. We approach this problem using two competing algorithmic strategies. The first, developed within the stochastic programming literature, is stochastic dual dynamic programming (SDDP) which uses Benders decomposition to create a multidimensional val…
We define a regularized variant of the Dual Dynamic Programming algorithm called REDDP (REgularized Dual Dynamic Programming) to solve nonlinear dynamic programming equations. We extend the algorithm to solve nonlinear stochastic dynamic programming equations. The corresponding algorithm, called SDDP-REG, can be seen as an extension of a regularization of the Stochastic Dual Dynamic Programming (SDDP) algorithm recently introduced which was studied for linear problems only and with less general prox-centers. We show the convergence of REDDP and SDDP-REG. We assess the performance of REDDP and …
A stagewise decomposition algorithm called value function gradient learning (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approach has been successful for linear problems, nonlinear problems may suffer from the increasing size of e…
This paper describes a generation and transmission (G&T) expansion planning tool based on Benders decomposition and multistage stochastic optimization: (i) A MIP-based "investment module" determines a trial expansion plan; (ii) an SDDP-based "operation module" calculates the expected operation costs for the trial plan and Benders cuts to the investment module based on the expected marginal costs of the capacity constraints at the optimal solution. The integrated G&T planning approach is illustrated with a realistic planning study for the Bolivian power system.
Several attempts to dampen the curse of dimensionnality problem of the Dynamic Programming approach for solving multistage optimization problems have been investigated. One popular way to address this issue is the Stochastic Dual Dynamic Programming method (SDDP) introduced by Perreira and Pinto in 1991 for Markov Decision Processes. Assuming that the value function is convex (for a minimization problem), one builds a non-decreasing sequence of lower (or outer) convex approximations of the value function. Those convex approximations are constructed as a supremum of affine cuts. On continuous t…
We introduce an extension of Stochastic Dual Dynamic Programming (SDDP) to solve stochastic convex dynamic programming equations. This extension applies when some or all primal and dual subproblems to be solved along the forward and backward passes of the method are solved with bounded errors (inexactly). This inexact variant of SDDP is described both for linear problems (the corresponding variant being denoted by ISDDP-LP) and nonlinear problems (the corresponding variant being denoted by ISDDP-NLP). We prove convergence theorems for ISDDP-LP and ISDDP-NLP both for bounded and asymptotically …
In this paper, we study multistage stochastic mixed-integer nonlinear programs (MS-MINLP). This general class of problems encompasses, as important special cases, multistage stochastic convex optimization with non-Lipschitzian value functions and multistage stochastic mixed-integer linear optimization. We develop stochastic dual dynamic programming (SDDP) type algorithms with nested decomposition, deterministic sampling, and stochastic sampling. The key ingredient is a new type of cuts based on generalized conjugacy. Several interesting classes of MS-MINLP are identified, where the new algorit…
In this paper, we focus on a data-driven risk-averse multistage stochastic programming (RMSP) model considering distributional robustness. We optimize the RMSP over the worst-case distribution within an ambiguity set of probability distributions constructed directly from historical data samples. The proposed RMSP is intractable due to the multistage nested minimax structure in its objective function, so we reformulate it into a deterministic equivalent that contains a series of convex combination of expectation and conditional value at risk (CVaR), which can be solved by a customized stochasti…
We consider a microgrid where different prosumers exchange energy altogether by the edges of a given network. Each prosumer is located to a node of the network and encompasses energy consumption, energy production and storage capacities (battery, electrical hot water tank). The problem is coupled both in time and in space, so that a direct resolution of the problem for large microgrids is out of reach (curse of dimensionality). By affecting price or resources to each node in the network and resolving each nodal sub-problem independently by Dynamic Programming, we provide decomposition algorith…
We introduce an extension of Dual Dynamic Programming (DDP) to solve linear dynamic programming equations. We call this extension IDDP-LP which applies to situations where some or all primal and dual subproblems to be solved along the iterations of the method are solved with a bounded error (inexactly). We provide convergence theorems both in the case when errors are bounded and for asymptotically vanishing errors. We extend the analysis to stochastic linear dynamic programming equations, introducing Inexact Stochastic Dual Dynamic Programming for linear programs (ISDDP-LP), an inexact variant…
Multistage stochastic programming deals with operational and planning problems that involve a sequence of decisions over time while responding to realizations that are uncertain. Algorithms designed to address multistage stochastic linear programming (MSLP) problems often rely upon scenario trees to represent the underlying stochastic process. When this process exhibits stagewise independence, sampling-based techniques, particularly the stochastic dual dynamic programming (SDDP) algorithm, have received wide acceptance. However, these sampling-based methods still operate with a deterministic r…
In this paper, we consider a multistage expected utility maximization problem where the decision maker's utility function at each stage depends on historical data and the information on the true utility function is incomplete. To mitigate adverse impact arising from ambiguity of the true utility, we propose a maximin robust model where the optimal policy is based on the worst-case sequence of utility functions from an ambiguity set constructed with partially available information about the decision maker's preferences. We then show that the multistage maximin problem is time consistent when th…
We propose a new algorithm for solving multistage stochastic mixed integer linear programming (MILP) problems with complete continuous recourse. In a similar way to cutting plane methods, we construct nonlinear Lipschitz cuts to build lower approximations for the non-convex cost to go functions. An example of such a class of cuts are those derived using Augmented Lagrangian Duality for MILPs. The family of Lipschitz cuts we use is MILP representable, so that the introduction of these cuts does not change the class of the original stochastic optimization problem. We illustrate the application o…
